e39c91637337fc1afc54fe8e215f1493395601a3 - openbmc/hiomapd

commit	e39c91637337fc1afc54fe8e215f1493395601a3	[log] [tgz]
author	Suraj Jitindar Singh <sjitindarsingh@gmail.com>	Tue Mar 28 10:47:43 2017 +1100
committer	Suraj Jitindar Singh <sjitindarsingh@gmail.com>	Tue Apr 11 11:41:54 2017 +1000
tree	0af6bb466ed1ca0afcb03e568a3a98da5723c06d
parent	2dff340a846068f33a0de171a75d4187bc189ea3 [diff]

tree: 0af6bb466ed1ca0afcb03e568a3a98da5723c06d

README.md

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Intro

This document describes a protocol for host to BMC communication via the mailbox registers present on the Aspeed 2400 and 2500 chips. This protocol is specifically designed to allow a host to request and manage access to the flash with the specifics of how the host is required to control this described below.

Version

Both version 1 and version 2 of the protocol are described below with version 2 specificities represented with V2 in brackets - (V2).

Problem Overview

"mbox" is the name we use to represent a protocol we have established between the host and the BMC via the Aspeed mailbox registers. This protocol is used for the host to control the flash.

Prior to the mbox protocol, the host uses a backdoor into the BMC address space (the iLPC-to-AHB bridge) to directly manipulate the BMCs own flash controller.

This is not sustainable for a number of reasons. The main ones are:

Every piece of the host software stack that needs flash access (HostBoot, OCC, OPAL, ...) has to have a complete driver for the flash controller, update it on each BMC generation, have all the quirks for all the flash chips supported etc... We have 3 copies on the host already in addition to the one in the BMC itself.
There are serious issues of access conflicts to that controller between the host and the BMC.
It's very hard to support "BMC reboots" when doing that
It's slow
Last but probably most important, having that backdoor open is a security risk. It means the host can access any address on the BMC internal bus and implant malware in the BMC itself. So if the host is a "bare metal" shared system in some kind of data center, not only the host flash needs to be reflashed when switching from one customer to another, but the entire BMC flash too as nothing can be trusted. So we want to disable it.

To address all these, we have implemented a new mechanism that we call mbox.

When using this mechanism, the BMC is solely responsible for directly accessing the flash controller. All flash erase and write operations are performed by the BMC and the BMC only. (We can allow direct reads from flash under some circumstances but we tend to prefer going via memory).

The host uses the mailbox registers to send "commands" to the BMC, which responds via the same mechanism. Those commands allow the host to control a "window" (which is the LPC -> AHB FW space mapping) that is either a read window or a write window onto the flash.

When set for writing, the BMC makes the window point to a chunk of RAM instead. When the host "commits" a change (via MBOX), then the BMC can perform the actual flashing from the data in the RAM window.

The idea is to have the LPC FW space be routed to an active "window". That window can be a read or a write window. The commands allow to control which window and which offset into the flash it maps.

A read window can be a direct window to the flash controller space (ie. 0x3000_0000) or it can be a window to a RAM image of a flash. It doesn't have to be the full size of the flash per protocol (commands can be use to "slide" it to various parts of the flash) but if its set to map the actual flash controller space at 0x3000_0000, it's probably simpler to make it the full flash. The host makes no assumption, it's your choice what to provide. The simplest implementation is to just route to the flash read/only.
A write window has to be a chunk of BMC memory. The minimum size is not defined in the spec, but it should be at least one block (4k for now but it should support larger block sizes in the future). When the BMC receive the command to map the write window at a given offset of the flash, the BMC should copy that portion of the flash into a reserved memory buffer, and modify the LPC mapping to point to that buffer.

The host can then write to that window directly (updating the BMC memory) and send a command to "commit" those updates to flash.

Finally there is a RESET_STATE. It's the state in which the bootloader in the SEEPROM of the POWER9 chip will find what it needs to load HostBoot. The details are still being ironed out: either mapping the full flash read only or reset to a "window" that is either at the bottom or top of the flash. The current implementation resets to point to the full flash.

Where is the code?

The mbox userspace is available on GitHub This is Apache licensed but we are keen to see any enhancements you may have.

The kernel driver is still in the process of being upstreamed but can be found in the OpenBMC Linux kernel staging tree:

https://github.com/openbmc/linux/commit/85770a7d1caa6a1fa1a291c33dfe46e05755a2ef

Building

The autotools of this requires the autoconf-archive package for your system

The Hardware

The Aspeed mailbox consists of 16 (8 bit) data registers see Layout for their use. Mailbox interrupt enabling, masking and triggering is done using a pair of control registers, one accessible by the host the other by the BMC. Interrupts can also be raised per write to each data register, for BMC and host. Write tiggered interrupts are configured using two 8 bit registers where each bit represents a data register and if an interrupt should fire on write. Two 8 bit registers are present to act as a mask for write triggered interrupts.

Layout

Byte 0: COMMAND
Byte 1: Sequence
Byte 2-12: Arguments
Byte 13: Response code
Byte 14: Host controlled status reg
Byte 15: BMC controlled status reg

Low Level Protocol Flow

What we essentially have is a set of registers which either the host or BMC can write to in order to communicate to the other which will respond in some way. There are 3 basic types of communication.

Commands sent from the Host to the BMC
Responses sent from the BMC to the Host in response to commands
Asyncronous events raised by the BMC

General Use

Messages usually originate from the host to the BMC. There are special cases for a back channel for the BMC to pass new information to the host which will be discussed later.

To initiate a request the host must set a command code (see Commands) into mailbox data register 0. It is also the hosts responsibility to generate a unique sequence number into mailbox register 1. After this any command specific data should be written (see Layout). The host must then generate an interrupt to the BMC by using bit 0 of its control register and wait for an interrupt on the response register. Generating an interrupt automatically sets bit 7 of the corresponding control register. This bit can be used to poll for messages.

On receiving an interrupt (or polling on bit 7 of its Control Register) the BMC should read the message from the general registers of the mailbox and perform the necessary action before responding. On responding the BMC must ensure that the sequence number is the same as the one in the request from the host. The BMC must also ensure that mailbox data regsiter 13 is a valid response code (see Responses). The BMC should then use its control register to generate an interrupt for the host to notify it of a response.

Asynchronous BMC to Host Events

BMC to host communication is also possible for notification of events from the BMC. This requires that the host have interrupts enabled on mailbox data register 15 (or otherwise poll on bit 7 of mailbox status register 1). On receiving such a notification the host should read mailbox data register 15 to determine the event code which was set by the BMC (see BMC Event notifications in Commands for detail). Events which are defined as being able to be acknowledged by the host must be with a BMC_EVENT_ACK command.

High Level Protocol Flow

When a host wants to communicate with the BMC via the mbox protocol the first thing it should do it call MBOX_GET_INFO in order to establish the protocol version which each understands. Before this the only other commands which are allowed are RESET_STATE and BMC_EVENT_ACK.

After this the host can open and close windows with the CREATE_READ_WINDOW, CREATE_WRITE_WINDOW and CLOSE_WINDOW commands. Creating a window is how the host requests access to a section of flash. It is worth noting that the host can only ever have one window that it is accessing at a time - hence forth referred to as the active window.

When the active window is a write window the host can perform MARK_WRITE_DIRTY, MARK_WRITE_ERASED and WRITE_FLUSH commands to identify changed blocks and control when the changed blocks are written to flash.

Independently, and at any point not during an existing mbox command transaction, the BMC may raise raise asynchronous events with the host to communicate a change in state.

Version Negotiation

Given that a majority of command and response arguments are specified as a multiple of block size it is necessary for the host and BMC to agree on a protocol version as this determines the block size. In V1 it is hard coded at 4K and in V2 the BMC chooses and specifies this to the host as a response argument to MBOX_GET_INFO. Thus the host must always call MBOX_GET_INFO before any other command which specifies an argument in block size.

The host must tell the BMC the highest protocol level which it supports. The BMC will then respond with a protocol level. If the host doesn't understand the protocol level specified by the BMC then it must not continue to communicate with the BMC. Otherwise the protocol level specified by the BMC is taken to be the protocol level used for further communication and can only be changed by another call to MBOX_GET_INFO. The BMC should use the request from the host to influence its protocol version choice.

Window Management

In order to access flash contents the host must request a window be opened at the flash offset it would like to access. The host may give a hint as to how much data it would like to access or otherwise set this argument to zero. The BMC must respond with the lpc bus address to access this window and the window size. The host must not access past the end of the active window.

There is only ever one active window which is the window created by the most recent CREATE_READ_WINDOW or CREATE_WRITE_WINDOW call which succeeded. Even though there are two types of windows there can still only be one active window irrespective of type. A host must not write to a read window. A host may read from a write window and the BMC must guarantee that the window reflects what the host has written there.

A window can be closed by calling CLOSE_WINDOW in which case there is no active window and the host must not access the LPC window after it has been closed. If the host closes an active write window then the BMC must perform an implicit flush. If the host tries to open a new window with an already active window then the active window is closed (and implicitly flushed if it was a write window). If the new window is successfully opened then it is the new active window, if the command fails then there is no active window and the previous active window must no longer be accessed.

The host must not access an lpc address other than that which is contained by the active window. The host must not use write management functions (see below) if the active window is a read window or if there is no active window.

Write Management

The BMC has no method for intercepting writes that occur over the LPC bus. Thus the host must explicitly notify the BMC of where and when a write has occured. The host must use the MARK_WRITE_DIRTY command to tell the BMC where within the write window it has modified. The host may also use the MARK_WRITE_ERASED command to erase large parts of the active window without the need to write 0xFF. The BMC must ensure that if the host reads from an area it has erased that the read values are 0xFF. Any part of the active window marked dirty/erased is only marked for the lifetime of the current active write window and does not persist if the active window is closed either implicitly or explicitly by the host or the BMC. The BMC may at any time or must on a call to WRITE_FLUSH flush the changes which it has been notified of back to the flash, at which point the dirty or erased marking is cleared for the active window. The host must not assume that any changes have been written to flash unless an explicit flush call was successful, a close of an active write window was successful or a create window command with an active write window was successful - otherwise consistency between the flash and memory contents cannot be guaranteed.

The host is not required to perform an erase before a write command and the BMC must ensure that a write performs as expected - that is if an erase is required before a write then the BMC must perform this itself.

BMC Events

The BMC can raise events with the host asynchronously to communicate to the host a change in state which it should take notice of. The host must (if possible for the given event) acknowledge it to inform the BMC it has been received.

If the BMC raises a BMC Reboot event then the host must renegotiate the protocol version so that both the BMC and the host agree on the block size. A BMC Reboot event implies a BMC Windows Reset event. If the BMC raises a BMC Windows Reset event then the host must assume that there is no longer an active window - that is if there was an active window it has been closed by the BMC and if it was a write window then the host must not assume that it was flushed unless a previous explicit flush call was successful.

The BMC may at some points require access to the flash and the BMC daemon must set the BMC Flash Control Lost event when the BMC is accessing the flash behind the BMC daemons back. When this event is set the host must assume that the contents of the active window could be inconsistent with the contents of flash.

Protocol Definition

Commands

RESET_STATE          0x01
GET_MBOX_INFO        0x02
GET_FLASH_INFO       0x03
CREATE_READ_WINDOW   0x04
CLOSE_WINDOW         0x05
CREATE_WRITE_WINDOW  0x06
MARK_WRITE_DIRTY     0x07
WRITE_FLUSH          0x08
BMC_EVENT_ACK        0x09
MARK_WRITE_ERASED    0x0a	(V2)

Sequence

The host must ensure a unique sequence number at the start of a command/response pair. The BMC must ensure the responses to a particular message contain the same sequence number that was in the command request from the host.

Responses

SUCCESS		1
PARAM_ERROR	2
WRITE_ERROR	3
SYSTEM_ERROR	4
TIMEOUT		5
BUSY		6	(V2)
WINDOW_ERROR	7	(V2)

Description:

SUCCESS - Command completed successfully

PARAM_ERROR - Error with parameters supplied or command invalid

WRITE_ERROR - Error writing to the backing file system

SYSTEM_ERROR - Error in BMC performing system action

TIMEOUT - Timeout in performing action

BUSY - Daemon in suspended state (currently unable to access flash) - Retry again later

WINDOW_ERROR - Command not valid for active window or no active window - Try opening an appropriate window and retrying the command

Information

All multibyte messages are LSB first (little endian)
All responses must have a valid return code in byte 13

Commands in detail

Note in V1 block size is hard coded to 4K, in V2 it is variable and must be queried with GET_MBOX_INFO. Sizes and addresses are specified in either bytes - (bytes) or blocks - (blocks) Sizes and addresses specified in blocks must be converted to bytes by multiplying by the block size.

Command:
	RESET_STATE
	Implemented in Versions:
		V1, V2
	Arguments:
		-
	Response:
		-
	Notes:
		This command is designed to inform the BMC that it should put
		host LPC mapping back in a state where the SBE will be able to
		use it. Currently this means pointing back to BMC flash
		pre mailbox protocol. Final behavour is still TBD.

Command:
	GET_MBOX_INFO
	Implemented in Versions:
		V1, V2
	Arguments:
		V1:
		Args 0: API version

		V2:
		Args 0: API version

	Response:
		V1:
		Args 0: API version
		Args 1-2: default read window size (blocks)
		Args 3-4: default write window size (blocks)

		V2:
		Args 0: API version
		Args 1-2: default read window size (blocks)
		Args 3-4: default write window size (blocks)
		Args 5: Block size as power of two (encoded as a shift)

Command:
	GET_FLASH_INFO
	Implemented in Versions:
		V1, V2
	Arguments:
		-
	Response:
		V1:
		Args 0-3: Flash size (bytes)
		Args 4-7: Erase granule (bytes)

		V2:
		Args 0-1: Flash size (blocks)
		Args 2-3: Erase granule (blocks)

Command:
	CREATE_{READ/WRITE}_WINDOW
	Implemented in Versions:
		V1, V2
	Arguments:
		V1:
		Args 0-1: Window location as offset into flash (blocks)

		V2:
		Args 0-1: Window location as offset into flash (blocks)
		Args 2-3: Requested window size (blocks)

	Response:
		V1:
		Args 0-1: LPC bus address of window (blocks)

		V2:
		Args 0-1: LPC bus address of window (blocks)
		Args 2-3: Actual window size (blocks)
	Notes:
		Window location is always given as an offset into flash as
		taken from the start of flash - that is it is an absolute
		address.

		LPC bus address is always given from the start of the LPC
		address space - that is it is an absolute address.

		The requested window size is only a hint. The response
		indicates the actual size of the window. The BMC may
		want to use the requested size to pre-load the remainder
		of the request. The host must not access past the end of the
		active window.

		The requested window size may be zero. In this case the
		BMC is free to create any sized window but it must contain
		atleast the first block of data requested by the host. A large
		window is of course preferred and should correspond to
		the default size returned in the GET_MBOX_INFO command.

		If this command returns successfully then the window which the
		host requested is the active window. If it fails then there is
		no active window.

Command:
	CLOSE_WINDOW
	Implemented in Versions:
		V1, V2
	Arguments:
		V1:
		-

		V2:
		Args 0: Flags
	Response:
		-
	Notes:
		Closes the active window. Any further access to the LPC bus
		address specified to address the previously active window will
		have undefined effects. If the active window is a
		write window then the BMC must perform an implicit flush.

		The Flags argument allows the host to provide some
		hints to the BMC. Defined Values:
			0x01 - Short Lifetime:
				The window is unlikely to be accessed
				anytime again in the near future. The effect of
				this will depend on BMC implementation. In
				the event that the BMC performs some caching
				the BMC daemon could mark data contained in a
				window closed with this flag as first to be
				evicted from the cache.

Command:
	MARK_WRITE_DIRTY
	Implemented in Versions:
		V1, V2
	Arguments:
		V1:
		Args 0-1: Flash offset to mark from base of flash (blocks)
		Args 2-5: Number to mark dirty at offset (bytes)

		V2:
		Args 0-1: Window offset to mark (blocks)
		Args 2-3: Number to mark dirty at offset (blocks)

	Response:
		-
	Notes:
		The BMC has no method for intercepting writes that
		occur over the LPC bus. The host must explicitly notify
		the daemon of where and when a write has occured so it
		can be flushed to backing storage.

		Offsets are given as an absolute (either into flash (V1) or the
		active window (V2)) and a zero offset refers to the first
		block. If the offset + number exceeds the size of the active
		window then the command must not succeed.

Command
	WRITE_FLUSH
	Implemented in Versions:
		V1, V2
	Arguments:
		V1:
		Args 0-1: Flash offset to mark from base of flash (blocks)
		Args 2-5: Number to mark dirty at offset (bytes)

		V2:
		-

	Response:
		-
	Notes:
		Flushes any dirty/erased blocks in the active window to
		the backing storage.

		In V1 this can also be used to mark parts of the flash
		dirty and flush in a single command. In V2 the explicit
		mark dirty command must be used before a call to flush
		since there are no longer any arguments. If the offset + number
		exceeds the size of the active window then the command must not
		succeed.


Command:
	BMC_EVENT_ACK
	Implemented in Versions:
		V1, V2
	Arguments:
		Args 0:	Bits in the BMC status byte (mailbox data
			register 15) to ack
	Response:
		*clears the bits in mailbox data register 15*
	Notes:
		The host should use this command to acknowledge BMC events
		supplied in mailbox register 15.

Command:
	MARK_WRITE_ERASED
	Implemented in Versions:
		V2
	Arguments:
		V2:
		Args 0-1: Window offset to erase (blocks)
		Args 2-3: Number to erase at offset (blocks)
	Response:
		-
	Notes:
		This command allows the host to erase a large area
		without the need to individually write 0xFF
		repetitively.

		Offset is the offset within the active window to start erasing
		from (zero refers to the first block of the active window) and
		number is the number of blocks of the active window to erase
		starting at offset. If the offset + number exceeds the size of
		the active window then the command must not succeed.

BMC Events in Detail:

If the BMC needs to tell the host something then it simply writes to Byte 15. The host should have interrupts enabled on that register, or otherwise be polling it.

Bit Definitions:

Events which should be ACKed:

0x01: BMC Reboot
0x02: BMC Windows Reset (V2)

Events which cannot be ACKed (BMC will clear when no longer applicable):

0x40: BMC Flash Control Lost (V2)
0x80: BMC MBOX Daemon Ready (V2)

Event Description:

Events which must be ACKed: The host should acknowledge these events with BMC_EVENT_ACK to let the BMC know that they have been received and understood.

0x01 - BMC Reboot:
	Used to inform the host that a BMC reboot has occured.
	The host must perform protocol verison negotiation again and
	must assume it has no active window. The host must not assume
	that any commands which didn't respond as such succeeded.
0x02 - BMC Windows Reset: (V2)
	The host must assume that its active window has been closed and
	that it no longer has an active window. The host is not
	required to perform protocol version negotiation again. The
	host must not assume that any commands which didn't respond as such
	succeeded.

Events which cannot be ACKed: These events cannot be acknowledged by the host and a call to BMC_EVENT_ACK with these bits set will have no effect. The BMC will clear these bits when they are no longer applicable.

0x40 - BMC Flash Control Lost: (V2)
	The BMC daemon has been suspended and thus no longer
	controls access to the flash (most likely because some
	other process on the BMC required direct access to the
	flash and has suspended the BMC daemon to preclude
	concurrent access).
	The BMC daemon must clear this bit itself when it regains
	control of the flash (the host isn't able to clear it
	through an acknowledge command).
	The host must not assume that the contents of the active window
	correctly reflect the contents of flash while this bit is set.
0x80 - BMC MBOX Daemon Ready: (V2)
	Used to inform the host that the BMC daemon is ready to
	accept command requests. The host isn't able to clear
	this bit through an acknowledge command, the BMC daemon must
	clear it before it terminates (assuming it didn't
	terminate unexpectedly).
	The host should not expect a response while this bit is
	not set.
	Note that this bit being set is not a guarantee that the BMC daemon
	will respond as it or the BMC may have crashed without clearing
	it.