.\" Copyright (c) 2013 Peter Grehan
.\" Copyright (c) 2015 xhyve developers
.\" All rights reserved.
.\"
.\" Redistribution and use in source and binary forms, with or without
.\" modification, are permitted provided that the following conditions
.\" are met:
.\" 1. Redistributions of source code must retain the above copyright
.\"    notice, this list of conditions and the following disclaimer.
.\" 2. Redistributions in binary form must reproduce the above copyright
.\"    notice, this list of conditions and the following disclaimer in the
.\"    documentation and/or other materials provided with the distribution.
.\"
.\" THIS SOFTWARE IS PROVIDED BY THE AUTHORS AND CONTRIBUTORS ``AS IS'' AND
.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
.\" ARE DISCLAIMED.  IN NO EVENT SHALL THE AUTHORS OR CONTRIBUTORS BE LIABLE
.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
.\" SUCH DAMAGE.
.\"
.Dd September 11, 2015
.Dt HYPERKIT 1
.Os
.Sh NAME
.Nm hyperkit
.Nd "run a guest operating system inside a virtual machine"
.Sh SYNOPSIS
.Nm
.Op Fl behuwxACHPWY
.Op Fl c Ar numcpus
.Op Fl g Ar gdbport
.Op Fl l Ar lpcdev Ns Op , Ns Ar conf
.Op Fl m Ar size Ns Op Ar K|k|M|m|G|g|T|t
.Op Fl p Ar vcpu:hostcpu
.Op Fl s Ar slot,emulation Ns Op , Ns Ar conf
.Op Fl U Ar uuid
.Op Fl f Ar firmware
.Sh DESCRIPTION
.Nm
is a hypervisor that runs guest operating systems inside a
virtual machine.
.Pp
Parameters such as the number of virtual CPUs, amount of guest memory, and
I/O connectivity can be specified with command-line parameters.
.Pp
.Nm
runs until the guest operating system reboots or an unhandled hypervisor
exit is detected.
.Sh OPTIONS
.Bl -tag -width 10n
.It Fl A
Generate ACPI tables.
Required for
.Fx Ns /amd64
guests.
.It Fl b
Enable a low-level console device supported by
.Fx
kernels compiled with
.Cd "device bvmconsole" .
This option will be deprecated in a future version.
.It Fl c Ar numcpus
Number of guest virtual CPUs.
The default is 1 and the maximum is 16.
.It Fl C
Include guest memory in core file.
.It Fl e
Force
.Nm
to exit when a guest issues an access to an I/O port that is not emulated.
This is intended for debug purposes.
.It Fl g Ar gdbport
For
.Fx
kernels compiled with
.Cd "device bvmdebug" ,
allow a remote kernel kgdb to be relayed to the guest kernel gdb stub
via a local IPv4 address and this port.
This option will be deprecated in a future version.
.It Fl h
Print help message and exit.
.It Fl H
Yield the virtual CPU thread when a HLT instruction is detected.
If this option is not specified, virtual CPUs will use 100% of a host CPU.
.It Fl l Ar lpcdev Ns Op , Ns Ar conf
Allow devices behind the LPC PCI-ISA bridge to be configured.
The only supported devices are the TTY-class devices
.Ar com1
and
.Ar com2.
.It Fl m Ar size Ns Op Ar K|k|M|m|G|g|T|t
Guest physical memory size in bytes.
.Pp
The size argument may be suffixed with one of K, M, G or T (either upper
or lower case) to indicate a multiple of kilobytes, megabytes, gigabytes,
or terabytes.
If no suffix is given, the value is assumed to be in megabytes.
.It Fl M
Print MAC address and exit if using vmnet.
.It Fl p Ar vcpu:hostcpu
Pin guest's virtual CPU
.Em vcpu
to
.Em hostcpu .
.It Fl P
Force the guest virtual CPU to exit when a PAUSE instruction is detected.
.It Fl s Ar slot,emulation Ns Op , Ns Ar conf
Configure a virtual PCI slot and function.
.Pp
.Nm
provides PCI bus emulation and virtual devices that can be attached to
slots on the bus.
There are 32 available slots, with the option of providing up to 8 functions
per slot.
.Bl -tag -width 10n
.It Ar slot
.Ar pcislot[:function]
.Ar bus:pcislot:function
.Pp
The
.Ar pcislot
value is 0 to 31.
The optional
.Ar function
value is 0 to 7.
The optional
.Ar bus
value is 0 to 255.
If not specified, the
.Ar function
value defaults to 0.
If not specified, the
.Ar bus
value defaults to 0.
.It Ar emulation
.Bl -tag -width 10n
.It Li hostbridge | Li amd_hostbridge
.Pp
Provide a simple host bridge.
This is usually configured at slot 0, and is required by most guest
operating systems.
The
.Li amd_hostbridge
emulation is identical but uses a PCI vendor ID of
.Li AMD .
.It Li virtio-net
Virtio network interface.
.It Li virtio-blk
Virtio block storage interface.
.It Li virtio-rnd
Virtio RNG interface.
.It Li virtio-9p
Virtio 9P backend bridge.
.It Li ahci-cd
AHCI controller attached to an ATAPI CD/DVD.
.It Li ahci-hd
AHCI controller attached to a SATA hard-drive.
.It Li uart
PCI 16550 serial device.
.It Li lpc
LPC PCI-ISA bridge with COM1 and COM2 16550 serial ports.
The LPC bridge emulation can only be configured on bus 0.
.El
.It Op Ar conf
This optional parameter describes the backend for device emulations.
If
.Ar conf
is not specified, the device emulation has no backend and can be
considered unconnected.
.Pp
Network devices:
.Bl -tag -width 10n
.It Ar tapN Ns Op , Ns Ar mac=xx:xx:xx:xx:xx:xx
.It Ar vmnetN Ns Op , Ns Ar mac=xx:xx:xx:xx:xx:xx
.Pp
If
.Ar mac
is not specified, the MAC address is derived from a fixed OUI and the
remaining bytes from an MD5 hash of the slot and function numbers and
the device name.
.Pp
The MAC address is an ASCII string in
.Xr ethers 5
format.
.El
.Pp
Block storage devices:
.Bl -tag -width 10n
.It Pa /filename Ns Oo , Ns Ar block-device-options Oc
.It Pa /dev/xxx Ns Oo , Ns Ar block-device-options Oc
.It file:///filename.qcow, Ns format=qcow Ns Oo , Ns Ar block-device-options Oc
.El
.Pp
The
.Ar block-device-options
are:
.Bl -tag -width 8n
.It Li nocache
Open the file with
.Dv O_DIRECT .
.It Li direct
Open the file using
.Dv O_SYNC .
.It Li ro
Force the file to be opened read-only.
.It Li sectorsize= Ns Ar logical Ns Oo / Ns Ar physical Oc
Specify the logical and physical sector sizes of the emulated disk.
The physical sector size is optional and is equal to the logical sector size
if not explicitly specified.
.It format=qcow
Indicates that the image is in qcow format. The filename must be
given in the URL syntax.
.El
.Pp
9P backend:
.Bl -tag -width 10n
.It port= Ns Ar port Ns Oo , Ns Ar 9p-device-options Oc
Connect to server listening on the loopback device at the given port
.It path= Ns Ar path Ns Oo , Ns Ar 9p-device-options Oc
Connect to server listening on the given Unix domain socket
.El
.Pp
The
.Ar 9p-device-options
are:
.Bl -tag -width 8n
.It Li tag= Ns Ar mount-tag Ns
The 9p mount tag
.El
.Pp
TTY devices:
.Bl -tag -width 10n
.It Li stdio Ns Oo , Ns Ar uart-device-options Oc
Connect the serial port to the standard input and output of
the
.Nm
process.
.It Pa /dev/xxx Ns Oo , Ns Ar uart-device-options Oc
Use the host TTY device for serial port I/O.
.It Pa autopty Ns Oo , Ns Ar uart-device-options Oc
Automatically select a host TTY device.
.It Pa autopty=/path/to/symlink Ns Oo , Ns Ar uart-device-options Oc
Automatically select a host TTY device and setup a convenience symlink.
.El
.Pp
The
.Ar uart-device-options
are:
.Bl -tag -width 8n
.It Pa log=/path/to/ring-log
Also log output to a ring buffer
.El
.It Fl u
RTC keeps UTC time.
.It Fl U Ar uuid
Set the universally unique identifier
.Pq UUID
in the guest's System Management BIOS System Information structure.
By default a UUID is generated from the host's hostname and
.Ar vmname .
.It Fl w
Ignore accesses to unimplemented Model Specific Registers (MSRs).
This is intended for debug purposes.
.It Fl W
Force virtio PCI device emulations to use MSI interrupts instead of MSI-X
interrupts.
.It Fl x
The guest's local APIC is configured in x2APIC mode.
.It Fl Y
Disable MPtable generation.
.It Fl f Ar firmware,arg1,arg2,arg3
.Pp
Firmware/bootrom configuration.
.Bl -tag -width 10n
.It kexec , Ns Pa kernel Ns Oo , Ns Pa ramdisk Ns Oc Ns Oo , Ns Pa cmdline Oc
Launch
.Ar kernel
using the Linux kexec protocol.
.It fbsd , Ns Pa userboot , Ns Pa bootvolume Ns Oo , Ns Pa kernelenv Ns Oc
Boot using the fbsd protocol.
.It bootrom , Ns Pa path
Boot using
.Ar path
as a firmware image. Allows booting a UEFI/tianocore binary, such as
the one from bhyve (source: <https://github.com/freebsd/uefi-edk2.git>; binary <https://people.freebsd.org/~grehan/bhyve_uefi/>).
.It multiboot , Ns Pa kernel Ns Oo , Ns Pa module Ns Oo ;cmdline Ns Oc Ns Oo :module Ns Oo ;cmdline Ns Oc Ns Oc Ns ... Oc Ns Oo , Ns Pa cmdline Ns Oc
Boot using the multiboot protocol.
.El
.Pp
Note that commas are required even for empty options with the exception that they may be omitted for any trailing empty options.
.El
.Sh EXAMPLES
To run a virtual machine with 1GB of memory, two virtual CPUs, a virtio
block device backed by the
.Pa /my/image
filesystem image, and a serial port for the console:
.Bd -literal -offset indent
hyperkit -c 2 -s 0,hostbridge -s 1,lpc -s 2,virtio-blk,/my/image \\
  -l com1,stdio -A -H -P -m 1G \\
  -f kexec,vmlinuz,initrd.gz,"earlyprintk=serial console=ttyS0"
.Ed
.Pp
Run a 24GB single-CPU virtual machine with three network ports, one of which
has a MAC address specified:
.Bd -literal -offset indent
hyperkit -s 0,hostbridge -s 1,lpc -s 2:0,virtio-net,tap0 \\
  -s 2:1,virtio-net,tap1 \\
  -s 2:2,virtio-net,tap2,mac=00:be:fa:76:45:00 \\
  -s 3,virtio-blk,/my/image -l com1,stdio \\
  -A -H -P -m 24G -f fbsd,userboot.so,bootvolume.img \\
.Ed
.Pp
Run an 8GB quad-CPU virtual machine with 8 AHCI SATA disks, a raw AHCI SATA volume,
an AHCI ATAPI CD-ROM, a single virtio network port, an AMD hostbridge,
and the console port connected to an
.Xr nmdm 4
null-modem device.
.Bd -literal -offset indent
hyperkit -c 4 \\
  -s 0,amd_hostbridge -s 1,lpc \\
  -s 1:0,ahci-hd,/images/disk.1 \\
  -s 1:1,ahci-hd,/images/disk.2 \\
  -s 1:2,ahci-hd,/images/disk.3 \\
  -s 1:3,ahci-hd,/images/disk.4 \\
  -s 1:4,ahci-hd,/images/disk.5 \\
  -s 1:5,ahci-hd,/images/disk.6 \\
  -s 1:6,ahci-hd,/images/disk.7 \\
  -s 1:7,ahci-hd,/images/disk.8 \\
  -s 1:8,ahci-hd,/dev/rdisk2 \\
  -s 2,ahci-cd,/images/install.iso \\
  -s 3,virtio-net,tap0 \\
  -l com1,/dev/nmdm0A \\
  -A -H -P -m 8G \\
  -f kexec,vmlinuz,initrd.gz,"earlyprintk=serial console=ttyS0"

.Ed

.Sh EXIT STATUS
The
.Nm hyperkit
process exits with one of the following values:
.Pp
.Bl -tag -width flag -compact
.It Li 0
The guest either powered down or halted cleanly.
.It Li 1
An error occurred.
.It Li 2
The guest rebooted/reset. The caller is responsible for starting a new hyperkit process in this case.
.It Li 3
A triple fault occurred.
.It Li 100
An invalid/unknown VM exit occurred.
.El

.Sh SIGNALS
.Bl -tag -width 10n
.It Pa SIGUSR1
Pause all VCPU threads
.It Pa SIGUSR2
Unpause all VCPU threads
.El

.Sh HISTORY
.Nm
is a port of FreeBSD's bhyve hypervisor to OS X that
works entirely in userspace and has no other dependencies.
.Sh AUTHORS
.An Michael Steil Aq Mt mist64@mac.com

.\ net ipc