Flash management
One of the main goal of Gapy is to help the user to produce the content of each flash.
Flash layout
The way the content of a flash is organized is specified through a JSON file describing all the possible sections that it can contain, and how they are organized together.
A default flash layout is provided for each target, so that by default, the user does not have to worry about this aspect. This default layout usually allows all possible kind of sections, so that all features are exposed. Please refer to the target documentation to know which is the default layout.
If there is a need to have a different layout, a custom layout can be provided for each flash. This can be useful for example to include several times the same file-system, or to include a dedicated section.
The flash layout JSON file can be specified with the following option:
gapy --flash-content=<filepath>@<flashname>
<filepath> is the path to the JSON file describing the flash layout and <flashname> is the name of the flash for which the layout should be changed. It is possible to give a different layout for each flash.
Flash layout format
Here is an exemple of a flash layout:
{
"sections": [
{ "name": "fsbl",
"template": "rom",
"properties": {
"binary": "lib/fsbl",
"boot" :" true"
}
},
{
"name": "meta_table",
"template": "meta table",
"properties": {
"ssbl_a": "ssbl_a",
"pt_a": "partition_table"
}
},
{
"name": "ssbl_a",
"template": "app binary",
"properties": {
"binary": "../../BUILD/GAP9_V2/GCC_RISCV_FREERTOS/ssbl"
}
},
{
"name": "partition_table",
"template": "partition table v2"
},
{
"name": "volume_table",
"template": "volume table",
"properties": {
"volumes": [
{
"name": "volume_app1",
"partitions": ["app1", "readfs"],
"bootable": true
},
{
"name": "volume_app2",
"partitions": ["app2", "lfs"],
"bootable": true
},
{
"name": "factory",
"partitions": ["partition_table","volume_table","meta_table", "fsbl"]
},
{
"name": "ssbl",
"partitions": ["ssbl_a"]
}
]
}
},
{
"name": "app1",
"template": "app binary",
"properties": {
"binary": "app_src/app1/BUILD/GAP9_V2/GCC_RISCV_FREERTOS/test"
}
},
{
"name": "app2",
"template": "app binary",
"properties": {
"binary": "app_src/app2/BUILD/GAP9_V2/GCC_RISCV_FREERTOS/test"
}
},
{
"name": "readfs",
"template": "readfs",
"properties": {
"files": ["app_src/app1/files/flash_file_0.bin", "app_src/app1/files/flash_file_1.bin"]
}
},
{
"name": "lfs",
"template": "lfs",
"properties": {
"size": 1048576,
"root_dir": null,
"img_path": "app_src/app2/BUILD/GAP9_V2/GCC_RISCV_FREERTOS/mram.bin.lfs"
}
}
]
}
The top object must contain the key section with an associated array containing an ordered list of sections of the flash. The sections will appear in the flash in the same order.
Each section is an object which must contain at least the keys name and template.
The key name must be associated a string describing the name of the section. This name can be used to refer to this section at various places, for example on the command-line. It can also be used for partition tables.
They key template must be associated a string describing which Gapy python class will handle this section. When a target is defined, each available flash is described, and the allowed sections for each flash are specified, associating the template name to a Python class. The template name can for example be rom, or readfs.
There can also be an optional key properties, in order to override the value of section properties. This key must be associated an object containing the definition of key-value properties. Each property is a key with an associated value in the object. The available properties are specific to each section, please refer to the section documentation to see what is allowed. The properties will be used by the Python class handling the section in order to collect additional information, such as files to be included in a file-system. These properties are defined by the section and can be overwritten by the flash layout, and also from command-line arguments, described in the next section.
Overwriting flash properties
Flash section properties can be overwritten from command-line arguments, like in this example:
gapy --flash-content=my_layout.json@flash --flash-property=file.bin@flash:readfs:files
As shown on this example, a new property value can be specified with the option –flash-property=<value>@<flashname>:<sectionname>:<propertyname>.
The type of the property value is the one of the default value specified when the section declared the property, and is used to properly convert the value specified on the command-line. If the value is a list, overwriting it will actually append the specified value to the list.
Overwriting properties can be useful to keep the flash layout as a template and then specify real values such as file-system files or ROM content on the command-line, depending on the application.
Available flash sections
Common Properties
- Following properties are common to all sections:
size: size in bytes of the section. The specified size is the size of the whole section, including headers. If supplied data is smaller than this size, then the section will be padded until reaching that size. If the supplied data is larger than this size, then an error will be raised.
ROM
A flash can often be used as a bootable device. In this case the ROM will expect a header in the flash describing the executable to be loaded in memory. This section can be included when the ROM is allowed to boot from this flash.
This section is chip-dependent. The only one currently available is
defined in gapylib.chips.gap.rom_v2 and can be used for all gap9 chips. This section must be placed
as the first section of the flash. It supports the following properties:
boot: its value is a boolean and should be set to true if the chip should boot on this flash. Gapy will upload the content of the section to the target only it is true. This is useful to avoid uploading the content while the target is booting through JTAG.
binary: its value is the path to the executable which should be loaded by the ROM. Gapy will use it to generate the required information in the ROM section. Note that if boot is false, the section content will not be uploaded, even though binary contains a valid executable.
xip_virtual_address XIP virtual address to be used for binary load.
xip_flash_address XIP physical (flash) address to be used for binary load.
xip_page_number Number of pages used by XIP.
xip_page_size Size of pages used by XIP, in the format for log(page_size) -9.
For the flash automatic mode, such a section is considered empty if property boot is set to false or if there is no executable specified.
Template name is : rom.
Application Binary
Similar section to the ROM section, but without ROM headers. This section is used to boot binaries via the SSBL. Using it save some flash memory by removing rom-specific metadata.
binary: its value is the path to the executable which should be loaded by the ROM. Gapy will use it to generate the required information in the app binary section.
For the flash automatic mode, such a section is considered empty if there is no executable specified.
Template name is : app binary. A partition table v2 and ssbl is required.
Meta table
This section must be placed directly after the first rom section. This section allows the potential first stage bootloader to find SSBL(s) and for applications and SSBLs to find page table(s).
This section has the following properties:
ssbl_a: primary ssbl section label
ssbl_b: secondary ssbl section label
pt_a: primary page table section label
pt_b: secondary page table section label
For the flash automatic mode, such a section is considered empty if all the next sections are considered empty.
Template name is: meta table. A partition table v2 is required, incompatible with partition table v1.
Partition table
This section can be placed anywhere in the flash and will build a partition table of all the sections which are after this one. This supports legacy/v1 partitioning scheme.
This section does not have any property.
For the flash automatic mode, such a section is considered empty if all the next sections are considered empty.
Template name is: partition table.
Partition table V2
This section can be placed anywhere in the flash and will build a partition table of all the sections in the flash. It supports V2 partition scheme (which includes volume tables and fallback SSBLs).
This section does not have any property.
For the flash automatic mode, such a section is considered empty if all other sections are considered empty.
In case of multi flash layout, this section needs to be uniquely positionned in internal flash. It will list all sections in every flashes.
Template name is: partition table v2.
Read FS
This section will generate a PMSIS read FS, which is a very simple read-only file-system, useful for input stimuli.
It supports the following property:
files: its value is a list of strings, each one specifying the path to a file to be embedded into the file-system.
For the flash automatic mode, such a section is considered empty if there is no file specified.
Template name is: readfs.
Little FS
This section will generate a LittleFS section.
It supports the following properties:
root_dir: string specifying the path to a directory on the workstation to be embedded inside the file-system. Only the content is included, not the directory itself. The content is copied into the section, and is available at runtime on the target.
img_path: string specifying the path to a pre-built Little FS image.
For the flash automatic mode, such a section is considered empty if there is no root directory specified.
Template name is: lfs.
Raw
This section is a placeholder for empty spaces. This is useful in order to make it visible in the partition table, for example to make the empty space at the end of the flash be visible from the partition table.
It supports the following properties:
size: size in bytes of the section. The size can be set to -1 in order to cover all the space from the beginning of the section to the end of the flash.
For the flash automatic mode, such a section is always considered empty.
Template name is : raw.
Volume Table
This section will generate a PMSIS volume table. This volume table will allow to associate a PMSIS useable label to layout-described sections, as well as group logically sections which compose an application or a given feature.
It supports the following property:
- volumes: its value is a list of volume descriptions. Each volume description contains the following properties:
name: the label by which PMSIS code will search for the volume.
partitions: a list of Gapy sections that belongs to this volume. To be noted: the label of the section will be the same as Gapy section name.
bootable: indicates whether this volumes contains an application binary.
boot_order: indicates the boot priority of a bootable volume.
{
"name": "volume_table",
"template": "volume table",
"properties": {
"volumes": [
{
"name": "volume_app1",
"partitions": ["app1", "readfs"],
"bootable": true,
"boot_order" : 0
},
{
"name": "volume_app2",
"partitions": ["app2", "lfs"],
"bootable": true,
"boot_order" : 1
},
{
"name": "factory",
"partitions": ["partition_table","volume_table","meta_table", "fsbl"]
},
{
"name": "ssbl",
"partitions": ["ssbl_a"]
}
]
}
}
In case of multi flash layout, this section needs to be uniquely positionned in internal flash. It may reference sections from any flash. If two sections in two flahes have the same name, then the first one found (based on order in which flashes were specified) will be used.
For the flash automatic mode, such a section is considered empty if there is no volume specified.
Template name is : volume table. A partition table v2 is required jointly with this section.
Displaying the flash section properties
The list of properties for each flash and each flash section can be obtained with this command:
gapy flash_properties
This should display something similar to this example:
Section properties for flash: mram
+-----------------+------------------------------------------------------------------------------------------------------------------+
| Section name | Section properties |
+-----------------+------------------------------------------------------------------------------------------------------------------+
| rom | +---------------+----------------+---------------------------------------------------+ |
| | | Property name | Property value | Property description | |
| | +---------------+----------------+---------------------------------------------------+ |
| | | binary | None | Executable to be loaded by the ROM. | |
| | | boot | False | True if the ROM will boot using this ROM section. | |
| | +---------------+----------------+---------------------------------------------------+ |
| partition table | |
| readfs | +---------------+----------------+---------------------------------------------+ |
| | | Property name | Property value | Property description | |
| | +---------------+----------------+---------------------------------------------+ |
| | | files | [] | List of files to be included in the ReadFS. | |
| | +---------------+----------------+---------------------------------------------+ |
| hostfs | +---------------+----------------+---------------------------------------------+ |
| | | Property name | Property value | Property description | |
| | +---------------+----------------+---------------------------------------------+ |
| | | files | [] | List of files to be included in the HostFS. | |
| | +---------------+----------------+---------------------------------------------+ |
| lfs | +---------------+----------------+---------------------------------------------------------------+ |
| | | Property name | Property value | Property description | |
| | +---------------+----------------+---------------------------------------------------------------+ |
| | | size | 0 | Size in bytes of the whole LittleFS. | |
| | | root_dir | None | Workstation directory content to be included in the LittleFS. | |
| | +---------------+----------------+---------------------------------------------------------------+ |
| raw | +---------------+----------------+-----------------------------------------------------------------------------+ |
| | | Property name | Property value | Property description | |
| | +---------------+----------------+-----------------------------------------------------------------------------+ |
| | | size | -1 | Size in bytes of the whole raw section (-1 to cover flash remaining space). | |
| | +---------------+----------------+-----------------------------------------------------------------------------+ |
+-----------------+------------------------------------------------------------------------------------------------------------------+
This is also showing the current value of the properties. If some properties are overwritten from the command-line, this will show the values after being overwritten so that the command-line can be checked using this command.
Displaying the flash layout
In order to better understand the layout of the flash, and especially in order to see the effect of overwritting section properties, the flash layout can be displayed with the following command:
gapy flash_layout
This should display something similar to this example:
+----------------+-----------------+--------------+-------------------------------------------------------------------------------------------+
| Section offset | Section name | Section size | Section content |
+----------------+-----------------+--------------+-------------------------------------------------------------------------------------------+
| 0x0 | rom | 0x4 | +--------+------+------------+---------------------------------------------+ |
| | | | | Offset | Size | Name | Content | |
| | | | +--------+------+------------+---------------------------------------------+ |
| | | | | 0x0 | 0x4 | ROM header | +--------+-----------------+------+-------+ | |
| | | | | | | | | Offset | Name | Size | Value | | |
| | | | | | | | +--------+-----------------+------+-------+ | |
| | | | | | | | | 0x0 | rom_header_size | 0x4 | 0x4 | | |
| | | | | | | | +--------+-----------------+------+-------+ | |
| | | | +--------+------+------------+---------------------------------------------+ |
| 0x4 | partition table | 0xa0 | +--------+------+-----------------+-----------------------------------------------------+ |
| | | | | Offset | Size | Name | Content | |
| | | | +--------+------+-----------------+-----------------------------------------------------+ |
| | | | | 0x4 | 0x20 | header | +--------+-------------------------+------+-------+ | |
| | | | | | | | | Offset | Name | Size | Value | | |
| | | | | | | | +--------+-------------------------+------+-------+ | |
| | | | | | | | | 0x4 | magic_number | 0x2 | 0x2ba | | |
| | | | | | | | | 0x6 | partition_table_version | 0x1 | 0x1 | | |
| | | | | | | | | 0x7 | nb_entries | 0x1 | 0x4 | | |
| | | | | | | | | 0x8 | crc | 0x1 | 0x0 | | |
| | | | | | | | | 0x9 | padding | 0xb | - | | |
| | | | | | | | | 0x14 | md5 | 0x10 | - | | |
| | | | | | | | +--------+-------------------------+------+-------+ | |
| | | | | 0x24 | 0x20 | section0 header | +--------+--------------+------+--------+ | |
| | | | | | | | | Offset | Name | Size | Value | | |
| | | | | | | | +--------+--------------+------+--------+ | |
| | | | | | | | | 0x24 | magic_number | 0x2 | 0x1ba | | |
| | | | | | | | | 0x26 | type | 0x1 | 0x1 | | |
| | | | | | | | | 0x27 | subtype | 0x1 | 0x81 | | |
| | | | | | | | | 0x28 | offset | 0x4 | 0xa4 | | |
| | | | | | | | | 0x2c | size | 0x4 | 0x8046 | | |
| | | | | | | | | 0x30 | name | 0x10 | - | | |
| | | | | | | | | 0x40 | flags | 0x4 | 0x0 | | |
| | | | | | | | +--------+--------------+------+--------+ | |
| | | | | 0x44 | 0x20 | section1 header | +--------+--------------+------+--------+ | |
| | | | | | | | | Offset | Name | Size | Value | | |
| | | | | | | | +--------+--------------+------+--------+ | |
| | | | | | | | | 0x44 | magic_number | 0x2 | 0x1ba | | |
| | | | | | | | | 0x46 | type | 0x1 | 0x1 | | |
| | | | | | | | | 0x47 | subtype | 0x1 | 0x81 | | |
| | | | | | | | | 0x48 | offset | 0x4 | 0x80ea | | |
| | | | | | | | | 0x4c | size | 0x4 | 0x0 | | |
| | | | | | | | | 0x50 | name | 0x10 | - | | |
| | | | | | | | | 0x60 | flags | 0x4 | 0x0 | | |
| | | | | | | | +--------+--------------+------+--------+ | |
| | | | | 0x64 | 0x20 | section2 header | +--------+--------------+------+--------+ | |
| | | | | | | | | Offset | Name | Size | Value | | |
| | | | | | | | +--------+--------------+------+--------+ | |
| | | | | | | | | 0x64 | magic_number | 0x2 | 0x1ba | | |
| | | | | | | | | 0x66 | type | 0x1 | 0x1 | | |
| | | | | | | | | 0x67 | subtype | 0x1 | 0x81 | | |
| | | | | | | | | 0x68 | offset | 0x4 | 0x80ea | | |
| | | | | | | | | 0x6c | size | 0x4 | 0x0 | | |
| | | | | | | | | 0x70 | name | 0x10 | - | | |
| | | | | | | | | 0x80 | flags | 0x4 | 0x0 | | |
| | | | | | | | +--------+--------------+------+--------+ | |
| | | | | 0x84 | 0x20 | section3 header | +--------+--------------+------+----------+ | |
| | | | | | | | | Offset | Name | Size | Value | | |
| | | | | | | | +--------+--------------+------+----------+ | |
| | | | | | | | | 0x84 | magic_number | 0x2 | 0x1ba | | |
| | | | | | | | | 0x86 | type | 0x1 | 0x1 | | |
| | | | | | | | | 0x87 | subtype | 0x1 | 0x81 | | |
| | | | | | | | | 0x88 | offset | 0x4 | 0x80ea | | |
| | | | | | | | | 0x8c | size | 0x4 | 0x1f7f16 | | |
| | | | | | | | | 0x90 | name | 0x10 | - | | |
| | | | | | | | | 0xa0 | flags | 0x4 | 0x0 | | |
| | | | | | | | +--------+--------------+------+----------+ | |
| | | | +--------+------+-----------------+-----------------------------------------------------+ |
| 0xa4 | readfs | 0x8046 | +--------+--------+--------------+----------------------------------------+ |
| | | | | Offset | Size | Name | Content | |
| | | | +--------+--------+--------------+----------------------------------------+ |
| | | | | 0xa4 | 0xc | header | +--------+----------+------+-------+ | |
| | | | | | | | | Offset | Name | Size | Value | | |
| | | | | | | | +--------+----------+------+-------+ | |
| | | | | | | | | 0xa4 | fs_size | 0x8 | 0x46 | | |
| | | | | | | | | 0xac | nb_files | 0x4 | 0x2 | | |
| | | | | | | | +--------+----------+------+-------+ | |
| | | | | 0xb0 | 0x1d | file0 header | +--------+-----------+------+--------+ | |
| | | | | | | | | Offset | Name | Size | Value | | |
| | | | | | | | +--------+-----------+------+--------+ | |
| | | | | | | | | 0xb0 | offset | 0x4 | 0x46 | | |
| | | | | | | | | 0xb4 | file_size | 0x4 | 0x4000 | | |
| | | | | | | | | 0xb8 | name_len | 0x4 | 0x11 | | |
| | | | | | | | | 0xbc | name | 0x11 | - | | |
| | | | | | | | +--------+-----------+------+--------+ | |
| | | | | 0xcd | 0x1d | file1 header | +--------+-----------+------+--------+ | |
| | | | | | | | | Offset | Name | Size | Value | | |
| | | | | | | | +--------+-----------+------+--------+ | |
| | | | | | | | | 0xcd | offset | 0x4 | 0x4046 | | |
| | | | | | | | | 0xd1 | file_size | 0x4 | 0x4000 | | |
| | | | | | | | | 0xd5 | name_len | 0x4 | 0x11 | | |
| | | | | | | | | 0xd9 | name | 0x11 | - | | |
| | | | | | | | +--------+-----------+------+--------+ | |
| | | | | 0xea | 0x4000 | file0 | +--------+------+--------+-------+ | |
| | | | | | | | | Offset | Name | Size | Value | | |
| | | | | | | | +--------+------+--------+-------+ | |
| | | | | | | | | 0xea | data | 0x4000 | - | | |
| | | | | | | | +--------+------+--------+-------+ | |
| | | | | 0x40ea | 0x4000 | file1 | +--------+------+--------+-------+ | |
| | | | | | | | | Offset | Name | Size | Value | | |
| | | | | | | | +--------+------+--------+-------+ | |
| | | | | | | | | 0x40ea | data | 0x4000 | - | | |
| | | | | | | | +--------+------+--------+-------+ | |
| | | | +--------+--------+--------------+----------------------------------------+ |
| 0x80ea | hostfs | 0x0 | |
| 0x80ea | lfs | 0x0 | +--------+------+------+---------+ |
| | | | | Offset | Size | Name | Content | |
| | | | +--------+------+------+---------+ |
| | | | +--------+------+------+---------+ |
| 0x80ea | raw | 0x1f7f16 | +--------+----------+--------+--------------------------------------+ |
| | | | | Offset | Size | Name | Content | |
| | | | +--------+----------+--------+--------------------------------------+ |
| | | | | 0x80ea | 0x1f7f16 | header | +--------+------+----------+-------+ | |
| | | | | | | | | Offset | Name | Size | Value | | |
| | | | | | | | +--------+------+----------+-------+ | |
| | | | | | | | | 0x80ea | data | 0x1f7f16 | - | | |
| | | | | | | | +--------+------+----------+-------+ | |
| | | | +--------+----------+--------+--------------------------------------+ |
+----------------+-----------------+--------------+-------------------------------------------------------------------------------------------+
It is also possible to tune the level of details with this option:
gapy --flash-layout-level=1 flash_layout
This will show the previous layout like this:
+----------------+-----------------+--------------+-------------------------------------+
| Section offset | Section name | Section size | Section content |
+----------------+-----------------+--------------+-------------------------------------+
| 0x0 | rom | 0x4 | +--------+------+------------+ |
| | | | | Offset | Size | Name | |
| | | | +--------+------+------------+ |
| | | | | 0x0 | 0x4 | ROM header | |
| | | | +--------+------+------------+ |
| 0x4 | partition table | 0xa0 | +--------+------+-----------------+ |
| | | | | Offset | Size | Name | |
| | | | +--------+------+-----------------+ |
| | | | | 0x4 | 0x20 | header | |
| | | | | 0x24 | 0x20 | section0 header | |
| | | | | 0x44 | 0x20 | section1 header | |
| | | | | 0x64 | 0x20 | section2 header | |
| | | | | 0x84 | 0x20 | section3 header | |
| | | | +--------+------+-----------------+ |
| 0xa4 | readfs | 0x8046 | +--------+--------+--------------+ |
| | | | | Offset | Size | Name | |
| | | | +--------+--------+--------------+ |
| | | | | 0xa4 | 0xc | header | |
| | | | | 0xb0 | 0x1d | file0 header | |
| | | | | 0xcd | 0x1d | file1 header | |
| | | | | 0xea | 0x4000 | file0 | |
| | | | | 0x40ea | 0x4000 | file1 | |
| | | | +--------+--------+--------------+ |
| 0x80ea | hostfs | 0x0 | |
| 0x80ea | lfs | 0x0 | +--------+------+------+ |
| | | | | Offset | Size | Name | |
| | | | +--------+------+------+ |
| | | | +--------+------+------+ |
| 0x80ea | raw | 0x1f7f16 | +--------+----------+--------+ |
| | | | | Offset | Size | Name | |
| | | | +--------+----------+--------+ |
| | | | | 0x80ea | 0x1f7f16 | header | |
| | | | +--------+----------+--------+ |
+----------------+-----------------+--------------+-------------------------------------+
The level can vary from 0 for minimal details to 3 for maximum details.
Flash automatic mode
For platforms like boards where the flash content needs to be uploaded, the target will automatically decide if the flash content must be uploaded when command flash is executed, depending on the content of flash.
For that, the flash will check the sections and will upload the content only if at least one section is not empty. It is up to each section to decide if it is empty depending on its properties. More information is available in each section documentation.
In case the default behavior is causing trouble, the upload operation can be forced by using the option –flash-no-auto.
Flash section definition
New sections can be defined by adding in Gapy a new class derived from gapylib.flash.FlashSection.
Here is an example of a section definition:
class RomFlashSection(FlashSection):
def __init__(self, parent: Flash, name: str, section_id: int):
super().__init__(parent, name, section_id)
self.segments = []
self.declare_property(name='binary', value=None,
description="Executable to be loaded by the ROM."
)
Section properties can be declared when the section is instantiated. Only the properties defined here can be overwritten by the flash layout or the command-line.
They can be used in other section methods to condition what the section is doing, by calling the
method gapylib.flash.FlashSection.get_property().
Adding more section properties is the best way to make the section behavior configurable from the command-line.
The section should also overload the method gapylib.flash.FlashSection.set_content(), which is called by the flash to give the
section its content. The section should then parse it and generate the section image out of that.
Some utilities are provided in gapylib.utils to declare C structures, which can be filled depending
on the section content, so that the flash can automatically generate the section image out of
the declared C structure. A simple example can be found in gapylib.fs.readfs.
Multi Flash layout
For multi flash layouts, it is required to write one json layout per flash. Syntax otherwise remains the same as single flash layouts. If wanting to use transparent multi flash accesses a partition table v2 section is required, with matching pi_fpv2 APIs in embedded code.
Warning
All required json layouts must be passed to Gapy at the same time with as many –flash-content=layout_flashX.json@flashX as needed. Flash images can not be created independently.
Both partition table v2 and volume table sections are natively able to search in multiple flashes, but these sections themselves must be located in the internal flash. Also, in target description, internal flash must be analyzed last, so that partition table v2 and volume table may search in other flashes.