SeptemberOS User Manual

Version 1.0 beta
Evaluation release

Daniel Drubin




Purpose

This user manual describes SeptemberOS Embedded RTOS as of beta version 1.0. It is intended for application writers and users of the product of any level. It explains interfaces for application code, system code, drivers and integrated components at level sufficient to use the RTOS as is and freely modify its core code and supplemented components in order to suit projects' needs.


Intended Audience and Reading Suggestions

This document is intended for users of SeptemberOS who will use it in embedded projects, distributors of the RTOS, development, support and technical writing personnel. Native users of SeptemerOS are embedded programmers, who are fluent in C programming language and embedded systems. This manual doesn't (on most occasions) provide detailed explainations of concepts and details of RTOS and embedded systems. It is assumed that the reader is familiar with these concepts, C programming language, make build system and GNU toolchain.

In most cases this manual also doesn't explain industry-standard interfaces implemented by SeptemberOS. Where relevant, the user is suggested references to external resources and specification that appear in the end of this manual.


Scope

SeptemberOS is a system component intended for use in embedded projects. It provides core OS (with task management, system resources management, complete hardware interfacing support) and common OS support libraries (networking, file systems access and software standards compatibility). SeptemberOS aims at standards compatibility for easy porting and application of existing knowledge, while maintaining small footprint for embedded systems, execution speed and real-time performance. Whenever the two directions conflict, SeptemberOS as a rule does not try to resolve it, but attempts to provide both options for the user to choose, according to every project's specific needs.

SeptemberOS is intended to take role of an embedded OS in many projects that would use embedded Linux today, but for which Linux'es performance is not satisfactory. There is a wide existing embedded/RTOS offer, which however doesn't present enough platform support, industry standards compliance or places too much technical and license limitation. SeptemberOS attempts to address problems of existing embedded/RTOS offer by providing industry standard APIs (standard C library, POSIX), hardware interfaces and complete buildable source code.

The OS is designed with aim to provide necessary common OS services to embedded projects and with concern in mind that embedded systems may need to tune, replace or remove OS components.


Perspective

SeptemberOS version 1.0 is a new embedded/RTOS product offer.

The OS provides components that may be used as building blocks for embedded application project and may be tuned or replaced individually. The following picture describes the idea.




SeptemberOS Building Blocks

The "Buildting Blocks" picture shows a general structure of SeptemberOS used in an embedded project. Higher blocks in the "building" are dependent on lower blocks. Every such "block" may be replaced by the user with a more appropriate implementation and without a need to modify anything else, as long as replacement block provides all the necessary interfaces. Unnecessary components may be completely left own (with all their dependants).



SeptemberOS Features

SeptemberOS is a real-time operating system which follows general design lines of dedicated embedded OS. Application is built with OS into a single binary; nothing else is needed to start. There is a single address space; multi-threaded tasking structure; no requirement for a filesystem to boot; no requirement for user intervention for booting and operation.

SeptemberOS offers a number of advantages over the most popular embedded Linux on the same platform: smaller footprint, better responsiveness, better performance, portability to 16-bit and 8-bit CPU/MCU. At the same time it offers a number of advantages over existing embedded OS including performance, modularity, standard interfaces and provision of complete buildable source code.

SeptemberOS is developed with the philosophy in mind that an embedded OS is a support library for the application's developer. It provides hardware interfaces (chip, board support, device drivers), task and memory management, task scheduling and synchronization. At the same time it is recognized that embedded developers must have fine control over OS; therefore SeptemberOS provides with complete source code and in modular form. It is easy and straight-forward to replace task management or memory management algorithms with custom ones, or to leave out task management at all for one-loop applications.

SeptemberOS doesn't use MMU features of CPUs, and is portable to MMU-less CPUs. It is written in C language, except for small chip-specific assembly parts (bootstrapping, IRQ handlers entry points and task switching); source code is intended to be easily understandable by a professional embedded developer.

Below a short features list is provided.

SeptemberOS is comprised of a number of core components; some of them are independent and provide base for other components and application code, and some of them are dependent on other components. The SeptemberOS components are: Device Manager, Memory Manager, Task Manager, Timers, Basic Services, Networking Support Library, File System Support Interface, Standard C Support Library, POSIX Support and Architecture Support Library. Sections below describe the components in details. The core components are available for all supported architectures and machines.

SeptemberOS is provided with a number of sample applications and device drivers. While they are not considered to be part of OS, they provide useful functionality and may be used as is or serve as basis for user's own drivers and applications. Drivers are part of implementation for specific platform, they are intended more to be used "as is", as there are OS components that depend on them. Sample applications are more intended to be used as basis, but may be also used as additional tasks or as is in user's designs (there are sample applications that implement command monitor, telnet server, HTTP server etc.)


Core Components

This chapter describes core components of SeptemberOS.

Device Manager

Device manager provides common initialization, deactivation and API for SeptemberOS device drivers. Device drivers are hardware interface support libraries which expose programming API to an application via standard access functions, special entry points for OS and interface to hardware in way of accessing and programming hardware resources and accepting interrupts via installed interrupt handlers.

It should be noticed that there is no requirement that device interface is written necessarily in a form of SeptemberOS device driver. SeptemberOS application runs in kernel mode and has the same privileges as OS components; all OS services are available to applications. It is possible to implement complete device interface including interrupt handling in application code.

Not all drivers fit within device manager’s model. For example, disk drivers and NIC drivers expose a completely different specific API and may choose not to implement device manager’s handlers.

Device manager addresses devices by device id and sub-device id. Both IDs take 16 bits and are combined into 32-bit value that designates device. Device and sub-device IDs are analogous to UNIX major and minor device numbers and serve the same purpose.

Device manager maintains static drivers configuration table. The table is defined in file config.h under special conditional compilation switch, which is defined by device manager source.

Configuration table defines device drivers loading order and entry points: init, deinit, read, write, ioctl, open and close. Below are specified prototypes of those entry points:

typedef int (*drv_init)(unsigned long id);
typedef int (*drv_deinit)(void);
typedef int (*drv_open)(unsigned sub_id);
typedef int (*drv_read)(unsigned sub_id, void *buffer, unsigned long length);
typedef int (*drv_write)(unsigned sub_id, const void *buffer, unsigned long length);
typedef int (*drv_ioctl)(unsigned sub_id, int cmd, va_list argp);
typedef int (*drv_close)(unsigned sub_id);
drv_init takes device ID as a parameter. All other functions take only sub-device ID as a parameter. All functions return negative value for error indication. drv_init, drv_open, drv_close and drv_deinit return 0 upon success; drv_read and drv_write are suggested to return number of bytes issued for read or write, respectively upon success; drv_ioctl may return any interface-specific non-negative value upon success.

It is recommended that additionally to returning negative number as error indication device drivers also set errno global variable to specific error code.

Device manager exposes initialization entry point to OS core:

void init_devman(void);

The initialization routine calls all drivers’ drv_init entry points in order in which drivers appear in configuration table. It is called by SeptemberOS initialization code during system start-up.

Driver’s drv_init handler is supposed to initialize the device and bring it to operational state.

void deinit_devman(void);


The de-initialization routine calls all drivers’ drv_deinit entry points in order in which drivers appear in configuration table. It is called by SeptemberOS shutdown initialization code during system bring-down.

Driver’s drv_deinit handler is supposed to de-initialize (or shutdown) the device and bring it to a state safe for power off or reset.

Device manager provides the following API functions that call respective driver’s handlers and return value that the driver’s handler returned.

int open_drv(unsigned long id);
int read_drv(unsigned long drv_id, void *buffer, unsigned long length);
int write_drv(unsigned long drv_id, const void *buffer, unsigned long length);
int ioctl_drv(unsigned long drv_id, int cmd, …);
int close_drv(unsigned long drv_id);

All API functions take combined device and sub-device ID (high 16 bits contain device ID and low 16 bits contain sub-device ID). The functions use device ID to index select driver from configuration table and call its entry point with sub-device ID as a parameter. Return value from those functions is value returned by the drivers, with the following exceptions. If a call was made referencing a non-existent driver, -1 is returned and errno is set to ENODEV. If and cases where the driver didn’t expose respective entry point (set NULL in appropriate entry in driver entry), -1 is returned and errno is set to EINVAL.

Driver's entry points are defined for convenience, but they are not mandatory. If a driver doesn’t need to do any per sub-device initialization during open, the application may skip calls to open_drv() and close_drv() and instantly call read_drv(), write_drv() or ioctl_drv().

The current SeptemberOS specification doesn’t connect device manager to POSIX support library’s I/O structures. Thus device manager is not dependent on inclusion of POSIX support library, but device drivers lack ability to report read and write events to tasks via select() events multiplexer.

Memory Manager

SeptemberOS memory manager implements standard C library functions: malloc(), calloc(), free() and realloc(). Please refer to the ISO C Standard (9899:1999) for specification of the functions, their parameters and semantics. All dynamic memory allocation in SeptemberOS is done with memory manager’s functions. OS components that use memory manager are: task manager, timers, network support library, file systems support library, standard C support library and POSIX support library.

Dynamic memory start address and size are defined in config.h (in architecture-dependent header) by constants DYN_MEM_START and DYN_MEM_SIZE.

Allocation routines shall satisfy alignment requirements of all its users. The most demanding requirement is specified in config.h (in architecture-dependent header) by constant BLOCK_ALIGN. Please consider alignment requirement of all components in your application before modifying the default value.

Task Manager

SeptemberOS task manager implements real-time task scheduler and task management services: task creation, suspension, waking, synchronization, event signalling. Task manager consists of two parts: Task manager uses Memory Manager’s, Timers and OS Basic services.

Task Management

SeptemberOS task manager operates on tasks via TASK_Q (task queue) structure. The structure contains task state and queue linking fields. Task queues need not any special handing by the client code, except for initialization of an empty queue to NULL for the first time. Task queues are priority queues: entries are sorted by tasks priorities. This doesn’t have specific meaning for runnable tasks (for which there is a separate queue for each priority level), and is intended for waiting queues that may contain tasks with different priorities.

Task manager provides the following API functions for task creation:

int start_task(TASK_ENTRY task_entry, unsigned priority, unsigned options, void *param);
int start_task_ex(TASK_ENTRY task_entry, unsigned priority, unsigned options, uintptr_t stack_base, uintptr_t stack_size,TASK_Q **ptask, void *param);

Both functions return 0 upon success and -1 for error. The second variant accepts stack base address and size for the new task, which should have been allocated by the caller. Besides, it returns pointer to the newly created task into output parameter ptask.

TASK_ENTRY is an entry point to a task, defined as below:

typedef void (*TASK_ENTRY)(void *param);

Maximum number of concurrently existing tasks is limited by a definition of MAX_TASKS in config.h
Start task functions may be called in any contexts. (Side note: it is not advisable to start high-priority tasks in interrupt handlers in order to perform interrupt bottom-half work. It is recommended that special high-priority tasks are created during system initialization, which will constantly wait for events from real interrupt handler).

Task manager provides the following API functions to terminate a task:

void terminate(void);
void end_task(TASK_Q *pq);

terminate() ends the currently running task, while end_task() ends any arbitrary task. end_task() doesn’t return an error, and has no effect if pq is not an existing task. Return from task’s entry function has the same effect as calling terminate(). terminate() shall be called only from within running task context. end_task() may be called from any context.

SeptemberOS task manager implements real-time task scheduling. There is a configurable number of priorities defined in config.h: NUM_PRIORITY_LEVELS. The task scheduler maintains a priority queue for runnable tasks on every priority level. The first task in the highest priority level’s queue will run.

Optionally, tasks on the same priority level may allow round-robin preemption among them. If a task has an option OPT_TIMESHARE set, then it will release CPU after running TICKS_PER_SLICE (defined in config.h) timer ticks and move to the end of its priority queue. The task will remain runnable and will not allow lower priority tasks to run.

Task priorities are set at creation time and may be later changed by the following API functions:

void set_task_priority(TASK_Q *task, unsigned priority);
void set_running_task_priority(unsigned priority);
The first function sets priority for any task, and the second function sets priority of the currently running task. set_running_task_priority() shall be called only within running task context. set_task_priority() may be called from any context.

Tasks may have options that affect their scheduling and state management. OPT_TIMESHARE option which we already met, if set, allows time-sharing with the tasks on the same priority level. OPT_FP option instructs task manager to save and restore FPU state when a task is switched to or from. Task options are set at creation time and may be later changed by the following API functions:

void set_task_options(TASK_Q *task, unsigned options);
void set_running_task_options(unsigned options);

The first function sets options for any task, and the second function sets options of the currently running task. set_running_task_options() shall be called only within running task context. set_task_options() may be called from any context.

Task manager provides the following API functions to manipulate task queues:

void enqueue_task(TASK_Q **queue, TASK_Q *task);
TASK_Q *dequeue_task(TASK_Q **queue);

enqueue_task() inserts a task into priority queue. It is inserted after all tasks with higher or equal priority and before the first task with lower priority. If queue was empty (NULL), then task becomes queue head. dequeue_task() removes the highest priority task from a queue. The removed task structure is returned. If the task was the only one in a queue, the queue becomes empty (NULL).

Removing a task from scheduler’s running queue has effect of making it not runnable. Inserting a task into scheduler’s running queue has effect of making it runnable. NOTE: those functions don’t perform actual task switch. enqueue_task() and dequeue_task() may be called in any context. It is not advisable to use those API functions anywhere except for task management code; applications and driver code should use other task management API functions.

Task manager provides the following API functions to suspend task and resume it:

void nap(struct task_q **waitq);
void wake(struct task_q **waking);

nap() makes running task “take a nap” (suspend). It receives a pointer to the head of waiting queue on which the task will wait. wake() wakes (makes runnable) the first task (with the highest priority) of the waiting queue. nap() shall be used only from within a running task context. wake() may be used from any context.

Synchronization Primitives

Task manager provides synchronization means: counting semaphores, mutexes and spinlocks. Semaphores, mutexes and spinlocks primitives are based on architecture-specific atomic read-modify-write operations in order to set a lock variable to “locked” state and retrieve its status in the same operation.

Semaphores and mutexes are implemented with waiting queues. If a calling task tries to acquire a busy semaphore or mutex, it will be put to wait. When the resources is released, the highest priority task from the waiting queue will wake up and take the resource. Semaphores and mutexes protection shall never be applied outside of running task context (e.g. in interrupt handlers). Spinlocks may be used in any context.

Semaphores are implemented within the following API functions:

void init_semaphore(SEMAPHORE *sema4, int max_count, int init_count);
void down(SEMAPHORE *sema4);
void up(SEMAPHORE *sema4);

init_semaphore() initializes semaphore to maximum count and initial count. down() tries to acquire a semaphore, up() releases it.

Mutexes are implemented within the following API functions:

void init_mutex(MUTEX *mutex);
void lock_mutex(MUTEX *mutex);
void unlock_mutex(MUTEX *mutex);

init_mutex() initializes a mutex. lock_mutex() tries to acquire a semaphore, unlock_mutex() releases it. Mutexes are semantically equivalent to semaphores with max_count 1.

Spinlocks are implemented with simple integer variables of type unsigned. The following API functions are used to acquire and release a spinlock, respectively.

void spin_lock(unsigned *lock_word);
void spin_unlock(unsigned *lock_word);

lock_word is an address of lock variable.

In many cases spinlocks may be used for mutual protection with higher-priority code (e.g. interrupt handler). The following convenience functions are available that acquire spinlock and mask a specific IRQ:

void spin_lock_irq(unsigned *lock_word, int irq, uint32_t *irq_mask);
void spin_unlock_irq(unsigned *lock_word, const uint32_t *irq_mask);

irq_mask is a pointer to IRQ mask bit array (size is platform-specific), which is saved by spin_lock_irq() and restored by spin_unlock_irq(). irq is interrupt number to mask.

Additionally there are two API functions to globally enable and disable task preemption:

void disable_preemption(void);
void enable_preemption(void);

Additionally, tasks may use disable_irqs(), enable_irqs() and other interrupts manipulation functions, which were related to Basic Services and Architecture Support Library.

Events

Task manager provides the following events waiting services:
  1. Simple events management.


    2. The following API functions allow a task initialize multiple events set, wait for events and retrieve specific events that were reported.

    void init_events_mul(EVENTS_MUL *ev);
    void wait_events_mul(EVENTS_MUL *ev);

    The structure EVENTS_MUL has the following definition:

    typedef struct events_mul
    {
    unsigned long events_mask; // Events of interest
    unsigned long current_events; // Current events (reported)
    struct task_q *wait_queue;
    } EVENTS_MUL;

    Prior to calling wait_events_mul() a task should set events_mask to desired events set. wait_queue is used to put a task to wait if one of selected events is not available at a time of call to wait_events_mul(). In order to post events to EVENTS_MUL structure, the following API call is used:

    void send_events_mul(EVENTS_MUL *ev, unsigned long events);

    Using the API above a task may wait for up to 32 events simultaneously. There is no distinct API for waiting for a single event. In a case of a single event the above API may be used, or if it is known that event didn’t occur at the time of interest, the waiting task may just call nap() and delivering code will wake it up using wake(). wait_events_mul() shall not be called outside of running task context. send_events_mul() may be called in any context.

  2. Select multiplexer events


    3. Task manager provides events multiplexer (intended for use by select() API function). The API is based on the following structure:

    typedef struct events_sel_q
    {
    int max_events; // Size of events arrays in bits
    unsigned long *pevents; // Current events (usually there will be only one event posted)
    unsigned long *events_mask; // Events of interest
    struct task_q *task; // This is actually a single task that waits on events selector queue
    struct events_sel_q *next, *prev;
    } EVENTS_SEL_Q;

    In EVENTS_SEL_Q structure task task queue is used to hold only a single waiting task. EVENTS_SEL_Q queue itself is sorted by task’s priority. events_mask is an array of unsigned integers that is set to desired events to watch. pevents is an array of unsigned integers that hold occurred events when a task waits. The following functions provide API for events multiplexer.

    void init_events_sel(EVENTS_SEL_Q *sel_q, unsigned long *pevents);

    Initializes EVENTS_SEL_Q.

    EVENTS_SEL_Q *new_events_sel(int max_events, unsigned long *events_mask);

    Allocates a new EVENTS_SEL_Q and sets its max_events and events_mask (the array is allocated, so the function may be called with automatic storage array as a parameter).

    void del_events_sel(EVENTS_SEL_Q *sel);

    Frees and disbands EVENTS_SEL_Q.

    void remove_events_sel(EVENTS_SEL_Q **sel_q, EVENTS_SEL_Q *p);

    Removes p from the queue sel_q.

    void wait_events_sel(EVENTS_SEL_Q **sel_q, EVENTS_SEL_Q *myself);

    If myself already contains reported events, return immediately. Otherwise, puts the running task to sleep in task queue inside myself events queue and inserts myself into events queue sel_q.

    void send_event_sel(EVENTS_SEL_Q **sel_q, int max_events, int event);

    Sends event to events queue.

    The following functions are convenience API to set, clear and retrieve a specific event in an array of events.

    void set_event(unsigned long *pevents, int max_events, int event);
    void clear_event(unsigned long *pevents, int max_events, int event);
    int is_event_set(unsigned long *pevents, int max_events, int event);

Timers

SeptemberOS provides with Timers API, which allows installation and removal of one-shot or periodic timers. Timers service depends on architecture or machine specific hardware timers, which are set up by Architecture Support Library’s initialization code. Also it depends on Memory Manager services and OS Basic Services.

Installed timers run in timer interrupt handler’s context; they have very small latency, but have restrictions natural to ISR code. Timers code should be considered as usual ISR code; in case that it needs to do some prolonged work the work should be placed in a high-priority task, which will receive wake-ups from the timer routine.

Timers service installs a handler for system timer interrupts and manages timers in multiples of system timer ticks. System timer tick is configured in config.h by TICKS_PER_SEC definition.

Timers are installed with the following API function:

int install_timer(timer_t *tm);

timer_t is defined as follows:

typedef struct timer
{
    long timeout; // Number of timer's ticks (this timer's ticks, see 'resolution').
    unsigned long latch; // Actual tick count.
    unsigned long resolution; // Timer's ticks per second. Cannot be more than system timer's resolution (TICKS_PER_SEC).
    unsigned flags; // Periodic or one-shot
    unsigned task_priority; // Task priority of a timer (tasks with greater priority will block this timer. Set to 0. - UNUSED
                    // in order to not allow any tasks mask out the timer's reporting
    timer_proc callback; // Callback (timer) function
    void *prm;    // Parameter that will be transferred to callback
} timer_t;


Timer routine has the following prototype:

typedef void (*timer_proc)(void *arg);

It is called with the same parameter that was transferred in prm field of timer_t structure when installing the timer. resolution parameter is intended for systems with multiple timers, where different software timers may be installed on different hardware timers. Currently this parameter has no real use and should be set to TICKS_PER_SEC. The timer routine will be called after number of ticks computable by the following formula: timeout * TICKS_PER_SEC / resolution.

A timer may be cancelled by calling to remove_timer(). This API function has the following prototype:

int remove_timer(timer_t *tm);

One-shot timers are removed automatically by the system after calling the timer handler. Timers may be installed and removed in any context, including timer routine.

Basic Services

This chapter specifies a small group of system core services, which don’t clearly belong to any OS component. They may be used by both system and application code.

Initialization

OS basics provide system initialization code. System initialization code calls initialization routines of all components in the following order:


After initializations are completed, the application’s entry point, app_entry() is called. It has the following prototype:

void app_entry(void);

Depending on whether START_APP_IN_TASK is defined in config.h, app_entry() is started in a context of initial task or as just a function out of any task context. The first case is suitable for easier start-up of an application, the second is suitable to prepare one-loop applications, possibly without task manager at all. If app_entry() is started in a task, it has priority INIT_TASK_PRIORITY and options INIT_TASK_OPTIONS (both are defined in config.h)

Interrupt Handling

OS basics provide an API call to set interrupt handler:

int set_int_callback(uint32_t irq_no, isr proc);

ISR has the following protptype:

typedef int (*isr)(void);

OS basics allow up to MAX_IRQ_HANDLERS (defined in config.h) to be installed for a single IRQ number. ISR shall return 0 in order to allow other (shared) interrupt handlers to process IRQ. ISR handlers run with the corresponding interrupt in “in service” state in interrupt controller and normally with interrupts disabled.

It may be convenient to expand the interface so that ISRs are supplied with void* parameter set at their installation time. This is generally intended to be pointer to device structure.

Networking Support Library

SeptemberOS provides Networking Support Library. The current specification includes glue to ethernet devices. The following sub-components comprise Networking Support Library:


Ethernet NIC

Ethernet NIC implements the following API functions:

void (*get_send_packet)(unsigned char **payload);
int (*send_packet)(unsigned char *dest_addr, word protocol, unsigned size);


get_send_packet() returns pointer to ethernet packet’s data area in payload. send_packet() sends a packet with payload contained in a buffer previously acquired by get_send_packet(). send_packet() takes destination ethernel address, protocol and size as parameters. send_packet() must be always called after a call to get_send_packet() and before another call to get_send_packet(). However, NIC driver functions need to provide locking in order to ensure correct sequence of calls. Generic ethernet module provides this locking functionality.

Ethernet NIC is responsible to call eth_parse_packet() in a context suitable for complete protocol parsing and possible data copying. The function has the following prototype:

void eth_parse_packet(struct net_if *this, void *pkt);

In general, all parsing functions receive pointer to network interface that received the packet and pointer to headers of this protocol, provided by lower-level protocol parsers.

Generic Ethernet Module

Generic ethernet module provides the following functionality. On parsing way ethernet module defines eth_parse_packet() function, which is called by ethernet NIC driver.

void eth_parse_packet(struct net_if *this, void *pkt);

On sending way it provides eth_get_send_packet() and eth_send_packet() functions. Their prototypes appear below:

void eth_get_send_packet(struct net_if *this, unsigned char **payload);
int eth_send_packet(struct net_if *this, unsigned char *dest_addr, word protocol, unsigned size);


Their prototypes are the same as NIC driver’s get_send_packet() and send_packet() functions, with addition of network interface pointer (from which correct driver is determined and called). Generic ethernet module’s counterpart eth_get_send_packet() and eth_send_packet() provide locking of the corresponding netowork interface in order to prevent another call to get_send_packet() before send_packet() corresponding to previous get_send_packet() was called. eth_get_send_packet() locks acquisition of send frames on the desired network interface and eth_send_packet() unlocks it.

In case that eth_get_send_packet() is called referencing locked network interface, payload is set to NULL on return. eth_send_packet() returns 0 if sending was successful and -1 if sending was attempted on a non-locked network interface.

ARP

Network Support Library provides with ARP functionality in order to resolve IP addresses to hardware Ethernet addresses for sending packets and let other ethernet hosts know hardware addresses of configured network interfaces. The ARP module provides the following interface for client code:

struct arp_tbl_entry *find_arp_entry(unsigned char *ip_addr);

This function finds ARP entry that correlates Ethernet address to requested IP address. The client code may use this function to determine whether an IP protocol-level packet may be sent without delay (returned is not NULL), or it will have to invoke arp_discover() in order to discover destination Ethernet address (NULL is returned).

int arp_discover(char *addr);

This function discovers Ethernet address that corresponds to requested IP address. It returns Boolean success indicator, 1 meaning that IP address was successfully resolved to Ethernet address and 0 means that no host resolved the IP address. arp_discover() uses ARP protocol to send broadcast messages, then it waits for replies. It may introduce up to several seconds delay to client code. If the function was called in a running task context, it will put the calling task to sleep. The function shall not be called in ISR context. If ISR code absolutely must send network packets, it should use direct Ethernet interface or ensure that the target Ethernet address has resolution to Ethernet address by calling find_arp_entry()/

Additionally the ARP module provides two callback functions to the lower-level Ethernet interface and IP interface.

void parse_arp(struct net_if *net_if, char *pdata);

This function is called by Ethernet packet parser when it finds that the packet is destined to ARP protocol. Depending on the context of the packet, the function either updates local ARP tables (ARP source IP address and Ethernet address present in the packet) or sends back ARP response (ARP packet contains request for a local IP address).

void update_arp_tbl(unsigned char *remote_ip_addr, struct eth_frame_hdr *frame_hdr);

This function is called for every IP packet received. If the correspondence between sending IP and Ethernet addresses doesn't exist in local ARP tables, the ARP tables are updated.

ARP module also provides API function to advertise itself using gratuitous ARP request to a specific network interface:

void send_grat_arp(struct net_if *net_if);

ARP module depends on Ethernet module, Task Manager, Timers and Memory Manager. At any time ARP module keeps at least one entry in local ARP tables, which correlates IP broadcast address to Ethernet broadcast address. Therefore, it is always safe to send broadcast messages without a risk to wait a number of seconds for ARP target address resolution.

IP

Network Support Library provides IPv4 implementation. IP protocol provides encapsulation for higher-level TCP, UDP and ICMP protocols and encapsulates itself into lower-level Ethernet frames.

IP module implements the following API for client code for sending packets:

struct net_if *get_net_interface(unsigned char *ip_addr);

The function get_net_interface is used to retrieve a network interface used to send data to requested IP address. Network interface may correspond to local IP address on the same network with the ip_addr or with default or configured gateway.

void prep_ip_hdr(struct ip_hdr *ip_hdr, word packet_len, byte protocol, const dword *src_ip, const dword *dest_ip);

The function prep_ip_hdr is used to prepare IP header. ip_hdr points to IP header area, the rest of parameters are placed in right fields of IP header; then IP header checksum is calculated and placed in IP header.

int ip_send_packet(struct net_if *net_if, unsigned char *ip_addr, unsigned size);

The function ip_send_packet is used to send packets to Ethernet layer. It accepts network interface in net_if parameter, target IP address in ip_addr and size including IP header. net_if parameter may be NULL, in which case the function will call retrieve network interface correct for the target IP address. The function retrieves Ethernet address for target IP (or gateway) from ARP module; if that is not available it will invoke arp_discover() in order to realize the target Ethernet address. ip_send_packet can be called safely in task context; if called in system or interrupt context the caller must ensure that target Ethernet address exists in local ARP tables by queried find_arp_entry() prior to calling ip_send_packet.

The general model of sending network packets provided by the IP module to higher protocol layers is the following:


NOTE: the client may choose to submit a packet directly to Ethernet interface in the last step by calling eth_send_packet instead of ip_send_packet.

The IP module provides two system entry points.

void parse_ip(struct net_if *net_if, char *pdata);

The function parse_ip is called by Ethernet parsing module's eth_parse_packet function when it discovers that the packet is designated to IP protocol.

void init_ip(void);

The function init_ip is called by system initialization code in order to allow IP module initialize its structures and state.

Interface for raw (IP) sockets is currently not implemented, but planned to be provided by IP module.

TCP

Network Support Library provides TCP implementation. TCP module offers reliable connection for bi-directional data transfer. It implements three-way connection handshake for client and server sides, simultaneous connections, independent two-way shutdown, sliding window acknowledgement and TCP retransmission timers. Please refer to TCP specification for details.

TCP module will reset connection (by sending a packet with RST flag set to peer) when a packet with wrong properties for the current connection's state is received, including segments with acknowledgement numbers more than window size above the last acknowledged segment. TCP module is closely related to Sockets module; all connection state, peer addresses information etc. is kept in sockets structures and may be accessed (in most cases) by Sockets API. TCP module provides interface to Sockets module (connect, accept connections, shutdown and reset connections, send and receive data) and to system (TCP parsing and initialization entry points).

The following API functions are provided to the client (Sockets module):

int tcp_connect(struct socket *psock, const struct sockaddr_in *address);

Connects socket represented by psock to address. The socket must be conforming with requirements placed on connecting sockets by Sockets API (must be of type SOCK_STREAM, non-listening and not connected, etc.). Please refer to Sockets API specification for details.

The function tcp_connect sends TCP SYN packet to peer and puts calling task to sleep on select() multiplexer. The task will wake up upon successful completion of TCP three-way connection handshake (or double two-way handshake for simultaneous connections) or with timeout error.

unsigned tcp_send(struct socket *psock, const void *message, size_t length, unsigned flags);

The function sends TCP data via the socket psock. The socket must conform to requirements placed on sockets of type SOCK_STREAM for sending data by Sockets API.

TCP module provides client with select multiplexer events. The following Sockets API calls receive select events: send(), sendto(), recv(), recvfrom(), connect(), accept(), select().

TCP module API functions shall be called only in running task context.

TCP module provides the following entry points to other system components:

void parse_tcp(struct net_if *net_if, char *pdata, struct ip_hdr *remote_ip_hdr);

The parse_tcp function parses TCP packet and performs necessary actions according to the designated socket's state and correctness of the packet. It is intended to be called by the IP module when the designated protocol in IP header is TCP. This function is responsible for delivery of read events to clients that wait on select multiplexer, which includes TCP sockets and write events to clients that wait on select for sockets to complete connection.

There is currently no TCP-specific initialization entry point to be called by system init code.

TCP module depends on IP, ARP and Ethernet modules, Memory Manager, Task Manager and Timers.

UDP

Network Support Library provides UDP implementation. UDP module offers unreliable datagram-based rata transfers (sends or receives). Please refer to UDP specification for details.

UDP module provides interface to Sockets module (send and receive data) and to system (UDP parsing and initialization entry points). Broadcast packets are supported for both send and receive. Multicast transfers are currently not supported.

The following API functions are provided to the client:

unsigned udp_send(struct socket *psock, const void *message, size_t length, const struct sockaddr_in *dest_addr, unsigned flags);


unsigned udp_send_to_netif(struct net_if *net_if, const void *message, size_t length, const struct sockaddr_in *src_addr, const struct sockaddr_in *dest_addr, unsigned flags);

The difference between two sending functions is that udp_send is used by Sockets support library and accepts pointer to socket structure psock as a parameter, and udp_send_to_netif is used to send a packet directly to network interface net_if when the caller is not Sockets library (it may be a system protocol which uses UDP for transport, such as DHCP).

udp_send must be called only in running task context, because it uses select() multiplexing. udp_send_to_netif may be used in any context if it was confirmed that the target address is found in local ARP tables or if the target address is broadcast.

UDP module provides client with select multiplexer events. The following Sockets API calls receive select events: send(), sendto(), recv(), recvfrom().

UDP module provides the following entry points to other system components:

void parse_udp(struct net_if *net_if, char *pdata, struct ip_hdr *remote_ip_hdr);

The parse_udp function parses UDP packet and performs necessary actions according to the designated socket's state. This function is responsible for delivery of read events to clients that wait on select multiplexer which includes UDP sockets.

There is currently no UDP-specific initialization entry point to be called by system init code.

UDP module depends on IP, ARP and Ethernet modules and Task Manager.

Sockets API

Network Support Library provides Berkeley Sockets API implementation. The following Sockets API functions are implemented: socket(), close(), bind(), accept(), connect(), getpeername(), getsockname(), getsockopt(), listen(), recv(), recvfrom(), send(), sendto(), setsockopt(), shutdown(), read(), write(), fcntl(), select(). Please refer to POSIX.1-2001 specification for detailed description of the functions prototypes and semantics.

Sockets module depends on TCP and UDP modules, Memory Manager, Task Manager and Standard C Support Library.

Interface for raw IP sockets is planned, but currently not implemented (will add dependency of Sockets module on IP module).

IP Address Conversion

Network Support Library provides the following IP address conversion API functions: inet_aton(), inet_addr() and inet_ntoa(). Please refer to POSIX.1-2001 specification for detailed description of the functions prototypes and semantics.

ICMP

Network Support Library provides minimal ICMP implementation in order to allow determination of network connection status of the SeptemberOS host by remote hosts with popular ping utility and to allow determination of network connection status of remote hosts by SeptemberOS application software via ping means. ICMP module implements ICMP Echo Request and ICMP Echo Response messages formatting, transfer and parsing.

ICMP provides the following entry points to other system modules:

void parse_icmp(struct net_if *net_if, char *pdata, struct ip_hdr *remote_ip_hdr);

The function parse_icmp is intended to be called by IP protocol parser when the designated protocol in IP header is ICMP. If Echo Request is received, response will be sent immediately.

ICMP module depends on IP module.

DHCP

The Network Support Library provides DHCP client implementation for dynamically configuring local IP address for the SeptemberOS host. DHCP uses UDP protocol in order to communicate with DHCP server (mostly via broadcast packets). Please refer to DHCP specification for details.

DHCP module provides the following interface for client code:

int dhcp_discover(struct net_if *net_if);

The function dhcp_discover initiates dynamic IP configuration of the specified interface. The configuration proceeds through DHCP states as responses arrive; necessary requests are determined from responses and are sent instantly in DHCP parsing context.

DHCP module provides the following interface to other system modules for parsing DHCP packets:

int parse_dhcp_client(struct net_if *net_if, char *pdata, size_t size);
int parse_dhcp_server(struct net_if *net_if, char *pdata, size_t size);

The functions parse_dhcp_client and parse_dhcp_server are called by the UDP module when no socket consumed a packet and the designated port in UDP header is DHCP client port or DHCP server port, respectively.

DHCP module is dependent on UDP and IP modules.

File Systems Support Library

File Systems Support Interface is provided for accessing file systems. Implementations will supply appropriate requests to access real file systems data structures on storage media or implement pseudo-filesystems. File Systems Support Interface depends on Disks numbering interface, which includes disk number and starting sector number. Disk and Starting Sector numbers are used by File Systems Support Interface to logically identify a file system; they need not necessarily correspond to physical disk storage media. File Systems Support Interface logically glues generic I/O layer to filesystems specific modules. In current implementation it connects POSIX I/O API to filesystems

struct fs

The File Systems Support Interface is accessible via the following interface structure.

struct fs
{
    int disk_num;    // Disk number
    int part_num;    // Partition number -- meanwhile unused --
    off_t start_offs;    // Starting offset of its partition
    char *mount_point;    // 'root' name for a FS, may be any string. (!) It's user's responsibility to assign distinct names
    void *fs_priv;    // Private structure specific to FS which must be instantiated
    int fs_type;    // ext2, fat, ...
    off_t dir_pos;    // For directory services
    int (*mount)(struct fs *this, const char *mount_point, int disk_num, unsigned start_sect);    // mount() entry point
    int (*unmount)(struct fs *this);    // unmount() entry point
    int (*file_open)(struct fs *this, const char *pathname, int flags, void **fs_entry);    // FS's generic open() procedure
    int (*file_creat)(struct fs *this, const char *pathname, mode_t mode, void **fs_entry);    // FS's generic creat() procedure
    ssize_t (*file_read)(struct fs *this, void *fs_entry, void *buf, off_t offs, size_t count);    // FS's generic read() procedure
    ssize_t (*file_write)(struct fs *this, void *fs_entry, const void *buf, off_t offs, size_t count);    // FS's generic write() procedure
    int (*file_close)(struct fs *this, void *fs_entry);    // FS's generic close() procedure
    int (*file_unlink)(struct fs *this, char *path);    // FS's generic unlink() procedure
    int (*file_rename)(struct fs *this, char *src_path, char *dest_path);    // FS's generic rename() procedure
    ssize_t (*get_file_size)(struct fs *this, void *fs_entry);
    unsigned long (*get_file_attrib)(struct fs *this, void *fs_entry);
    struct dirent *(*read_dir)(struct fs *this, void *fs_entry, off_t offs);
    size_t (*seek_dir)(struct fs *this, void *fs_entry, off_t offs);    // Returns correct offset for given offs - to be used in further read_dir().
    int (*stat)(struct fs *this, const char *path, struct stat *buf);    // stat()
    int (*fstat)(struct fs *this, void *fs_entry, struct stat *buf);    // fstat()
    void (*sync)(struct fs *this);    // Just sync()
};
 disk_num and start_offs are used to identify filesystem's data structures on the media. For pseudo-filesystems the fields may be ignored. Fields part_num, fs_priv, fs_type and dir_pos are reserved for internal use by the file system implementation. fs_type is set by FS implementation's mount() function and may be later tested by the FS management code in order to verify that the structure corresponds to a valid filesystem.

FS Implementation API Entries

mount entry point ignores all other fields of struct fs. It fills other structure's fields, including fs_type, mount_point and all API function pointers. mount_point is chosen by FS management code. It may be any scheme arbitraty chosen; filesystem must be just supplied mount point for mount() API call, which it will store in FS structure for the management code's reference. FS implementations refer file names related to their root directory in UNIX naming conventions ("/" is FS'es root directory, directory names are separated with "/").

All FS API entry points accept pointer to FS structure instance as the first parameters. File system entities are referenced via private structure fs_entry, which is returned in output parameter by file_open() method. This pointer cast as void* serves through all other file-related API calls as an analog to POSIX file descriptor or standard C library pointer to FILE.

API functions provide the following functionality:
mount() – mounts file system under given name and fills the file system structure.
unmount() – unmounts file system and frees all internal structures. The fs structure shouldn't be used as reference to a valid filesystem any more
file_open() – opens a file entity; pathname and flags parameters have the same meaning as in POSIX open(). fs_entry is filled with a void pointer which is used to identify the file entity afterwards, until closed
file_read() – reads from file entity. buf points to destination buffer, offs is an offset from beginning of file (file pointer), count is bytes count to read
file_write() – writes to file entity. buf points to source buffer, offs is an offset from beginning of file (file pointer), count is bytes count to read. File entity on filesystem may grow as result of file_write() request
file_close() – closes file system entity by releasing all associated structures and stamping last modification time if necessary. This make fs_entry no longer valid to reference a file entity object
file_unlink unlinks the file entity object from a filesystem. path is relative to the filesystem's root directory
file_rename changes name of file entity object, possibly moving it from one directory to another. src_path and dest_path are relative to the filesystem's root directory
get_file_size returns up-to-date size of a file entity as known in the filesystem
get_file_attrib returns up-to-date attributes of a file entity as known in the filesystem
read_dir reads directory record from filesystem at given offset. fs_entry must refer to a directory. Returns directory record read or NULL upon error. NOTE: subsequent calls to read_dir may return the same pointer but filled with different records. The caller should not assume that content of a directory record returned by one call to read_dir will remain after the next call
seek_dir seeks directory referred to by fs_entry to offs. Returns correct offset to be used by subsequent call to read_dir
stat, fstat, sync are counterparts for POSIX API calls stat(), fstat() and sync()

Standard C Support Library

Standard C Support Library provides convenience of using many of familiar standard C functions specified in the ISO C Standard (9899:1999). This chapter doesn't provide detailed descriptions of those functions, but mentions differences and provides list of supported functions. In general, most of standard C library is supported except for math functions.

Standard C Support Library provides the following functions: isspace(), isascii(), isdigit(), toupper(), tolower(), sprint(), vsprintf(), sscanf(), vsscanf(), memcpy(), memmove(), memset(), strcpy(), strncpy(), strcat(), strncat(), memcmp(), strcmp(), strcasecmp(), strcoll(), strncmp(), strxfrm(), memchr(), strchr(), strrchr(), strcspn(), strpbrk(), strspn(), strstr(), strtok(), strlen(), strerror(), malloc(), calloc(), free(), realloc(), random(), time(), localtime(), asctime(), ctime(), gmtime(), mktime(), sleep(), rename(), fopen(), freopen(), fclose(), fread(), fwrite(), fprintf(), printf(), vfprintf(), fscanf(), scanf(), vfscanf(), fgetc(), fgets(), getc(), ungetc(), gets(), fputs(), fputc(), putc(), setvbuf(), setbuf(), fflush(), feof(), ferror(), clearer(), fseek(), ftell(), rewind(), fgetpos(), fsetpos(), system()

Standard C Support Library provides the following standard C global data: errno variable of type int and stdin, stdout and stderr stream pointers of type FILE*.

Standard C Support Library provides initialization entry point for system start-up to call:

void init_libc(void);

C linrary functions may be called only after init_libc() has been called.

NOTE: it is recognized that math functions don't provide benefit for embedded systems in many cases. Often SeptemberOS software will run on CPU/MCU without FP support in hardware, and often without any need in math library. While it is justified to save on footprint on such systems, it will be examined in which cases and on which systems it is beneficial to include math support.

POSIX Support Library

POSIX Support Library provides convenience of using many of familiar standard POSIX functions specified in the POSIX Standard (POSIX.1-2001). This chapter doesn't provide detailed descriptions of those functions, but mentions differences and provides list of supported functions.

POSIX Support Library provides the following I/O functions: open(), creat(), read(), write(), lseek(), fcntl(), close(), sync(), fsync(), fdatasync(), unlink(), dup(), dup2(), link(), mknod(), stat(), fstat(), lstat(), select(), opendir(), closedir(), readdir(), rewinddir(), seekdir(), telldir(), pipe(). Sockets API functions, which are also part of POSIX.1-2001 standard are provided by the Network Support Library.

POSIX Support Library provides the following pthreads emulation API functions: pthread_create(), pthread_exit(), pthread_join(), pthread_detach(), pthread_kill(), pthread_attr_init(), pthread_attr_destroy(), pthread_equal(), pthread_attr_getdetachstate(), pthread_attr_setdetachstate(), pthread_attr_getschedparam, pthread_attr_setschedparam(), pthread_attr_getschedpolicy(), pthread_attr_setschedpolicy(), pthread_attr_getinheritsched(), pthread_attr_setinheritsched(), pthread_attr_getscope(), pthead_attr_setscope(), pthread_attr_getstack(), pthread_attr_setstack(), pthread_attr_getstacksize(), pthread_attr_setstacksize(), pthread_attr_getstackaddr(), pthread_attr_setstackaddr(), pthread_setschedprio().

POSIX support library provides the following process emulation API functions: execve(), execvp(), execv(), execle(), execlp(), exit(), atexit(), wait(), waitpid(), signal(), kill(), getpid(), getppid(), raise().

NOTE: exit(), atexit() and signal() are also standardized by ISO C Standard Library.

NOTE: fork() and vfork() functions are not supported because their semantics are not compatible with SeptemberOS memory architecrure.

POSIX support library supports the following types of file descriptors: regular files, devices, sockets, pipes, FIFOs. Devices are configured to have symbolic name corresponding to device ID and sub-device ID. There are no rules or limitations on what structure the names should have; the current implementation uses familiar "/dev/xxx" naming convention. There are no device nodes or other file system objects associated to device names.

POSIX support library provides the following initialization entry point for system init code:

void init_io(void);


Architecture Support Library

Architecture Support Library provides architecture and machine-dependent initializations, interrupt entry points and task switching back-ends. New architecture or machine ports will provide the interface specified in this chapter for the platform that they support.

Architecture Support Library is called internally by system initialization code, by CPU via hardware reporting and by system components such as the Task Manager in order to perform system-dependent task switching part.

Architecture Support Library provides the following system API functions:

void arch_eoi(int int_no);

Performs architecture and machine-specific eond-of-interrupt issue to interrupt controller

void switch_to(TASK_Q *pq);

Performs registers and CPU state context loading from pq. NOTE: intended for exclusive use by Task Manager during task switching

void switch_task(TASK_Q *pq);

Performs task switch (saves register and CPU state context in running tasks's structure, then performs switch_to(pq). NOTE: intended for exclusive use by Task Manager during task switching

void init_new_task_struct(TASK_Q *pq, TASK_ENTRY task_entry, dword param);

Initializes system-specific fields in task structure. NOTE: intended for exclusive use by Task Manager during task creation

Synchronization functions are provided by Architecture Support Library (platform-specific part of Task Manager).


Building and Debugging

This chapter explains configuration, building and debugging of SeptemberOS.

When you unpack SeptemberOS package, you will find the following sub-directories:

tools/ Tools specific to some targets
src/ SeptemberOS source, build and image directories

Source Tree Structure

SeptemberOS is provided in form of buildable source code. The tree is arranged in source and build target directories, with a single makefile. SeptemberOS is suited to be built with standard GNU toolchain, with all prefixes and definitions appearing in the makefile. Base source directory looks like the following:

arch/
base/
changelog.txt
drivers/
fs/
image/
include/
libc/
makefile
network/
obj/
posix/
samples/
changelog.txt lists the most recent changes.
makefile is used to build application with OS binary.

arch/ directory contains architecture-specific and platform-specific code and headers. This is mostly related to OS bootstrap, interrupt service entry points, platform initialization and shut down procedures and task initialization and platform-specific task switching (CPU registers, state etc.)  Under arch/ directory one can find sub-directories for specific architectures supported (currently those are x86/, arm/ and mips/). Next level, under each supported architecture there are directories for supported platforms (machines), that are named in form of mach-xxx. Currently supported machines are: pc for x86, versatile and evmdm6467 for arm and malta for mips. The mach-xxx sub-directories contain actual platform-specific code and headers.

base/ directory contains SeptemberOS core services. Below is list of files found in base/:

devman.c device manager
memman.c memory manager
sosbasic.c interrupt handler setting, calling installed ISRs, OS entry point
taskman.c task manager
timers.c timers

drivers/ directory contains SeptemberOS drivers. Note that drivers are included for all platforms; compilation for a specific platform selects also a specific drivers set. drivers/ directory includes the following sub-directories:

bus/ bus drivers: i2c, ide, pci, usb
keyboard/ PC keyboard
net/ NICs
serial/ UARTs
terminal/ PC text-mode VGA terminal

fs/ directory contains filesystems implementations. Currently there are ext2/ and fat/ subdirectories with ext2 and FAT12/FAT16 implementations.

NOTE: recently Microsoft began enforcing their patents on FAT (up to now concerning long file names implementation and FAT32 specifics). Following, SeptemberOS doesn't implement FAT with long file names and FAT32; we use as base and recommend where applicable to use ext2.

image/ directory contains image and supplement output resulting  from build.

include/ directory contains all generally usable headers except for platform-specific headers. Platform-specific headers reside in arch/$(ARCH)/mach-$(MACH)/include/. See "Building SeptemberOS Application" for settings of ARCH and MACH.

libc/ directory contains SeptemberOS implementation of a subset of standard C library

network/ directory contains implementation of network protocols and Sockets API

obj/ directory contains compiled objects

posix/ directory contains implementation of POSIX compatibility layer: POSIX I/O, pthreads and process emulation

samples/ directory contains sample SeptemberOS applications

Building SeptemberOS Applications

Host Environment and Targets

The supplied build system uses the following environment:
The x86 port is built into PE image, which is then bundled with standard bootsector into an image which can be copied to a boot device. x86-specific build includes the following tools: nasm, sed amd dd. The distribution includes versions of ld, objcopy, objdump, nm, strip and ar specific for PE image format for x86 targets.

Makefile

SeptemberOS doesn't mandate a build system. The build environment provided with the distribution is prepared for simplicity; it includes a straight forward build process, without automated tools or hidden steps. SeptemberOS applications and other software modules are built from a single makefile. There are no includes in makefile; all definitions, settings and targets are found in the makefile. The makefile is intended to be read and modified by humans, it includes comments for targets and definitions and understandable definitions and settings names. This way, the simple SeptemberOS build process can be easily incorporated as one of build steps of a larger project or modified to use automated build tools.

The makefile offers targets for all source files to compile. In order to build a sample application you may run `make app_name', e.g. `make http-srv' in order to build an  HTTP server, or `make uart-mon' in order to build a UART version of command monitor. `make clean' removes all resulting executables and object files and ensures complete rebuild the next time you choose a buildable target. If necessary, you may build every source file that is included in the project independently - look at target in makefile that relates to it.

All source files are compiled into objects in obj/ directory. This is intended for building one project for only one architecture and machine. We consider this case to be typical. If necessary, modify the makefile to compile sources into architecture and machine specific object directories and use them to complete architecture and machine specific output image. Apparently, with the current build arrangement it is necessary to `make clean' when compiling for a different architecture. This would not be necessary when architecture and machine specific build output directories are used.

You would need to modify the following makefile settings in order to customize build for your needs:

ARCH set to desired architecture
MACH set to desired platform (machine)
BUILD_TOOLS_PREFIX set to your toolchain's target-specific prefix (may depend on $(ARCH) and $(MACH)

The following settings you may want to modify in order to customize the build for your convenience:

CC C compiler
LD linker
AS assembler
ASM x86 assembler to build boot sector
OBJCOPY objcopy
OBJ_DIR directory to put object files (you may want to make this specific to architecture and machine
IMAGE_DIR directory to put object files (you may want to make this specific to architecture and machine
C_OPT C compiler options
A_OPT assembler options (x86)
L_OPT linker options

When porting to a different architecture and/or platform it is most easy to use the current source layout (add architecture to arch/, add platforms to arch/your-arch/,then modify makefile to add your architecture and platfroms similar to existing ones).

Object files are divided into several groups according to their purpose:

SEPTOS_OBJS Core SeptemberOS services
DRV_OBJS drivers objects
NET_OBJS Network support library objects
FS_OBJS Filesystems objects
LIBC_OBJS Standard C library objects
APP_OBJS Application objects

You may modify those according to your needs, or optionally leaved them blank if you don't need certain objects. See also Configuring SeptemberOS

Configuring SeptemberOS

SeptemberOS provides many configurable options. Some of them are specific to architecture / platform but most are customizable. Options fall into one of two categories:


Options configurable in makefile

There are several options configurable through makefile (except for architecture / platform and build tools specifics, which are described above).

CFG_MEMMAN set to non-empty definition in order to include dynamic memory manager. (NOTE: task manager and most other OS services, including many device drivers depend on it)
CFG_TASKMAN set to non-empty definition in order to include task manager. (NOTE: POSIX I/O and most other OS services, including many device drivers depend on it)
CFG_DEVMAN set to non-empty definition in order to include devices manager and SeptemberOS drivers infrastructure. (NOTE: POSIX I/O, timers and most other OS services, including all device drivers depend on it)
CFG_TIMERS set to non-empty definition in order to include installable timers. (NOTE: many OS services, including TCP/IP depend on it)
CFG_PTHREADS set to non-empty definition in order to include pthreads emulation
CFG_POSIXIO set to non-empty definition in order to include POSIX I/O compatibility layer
CFG_POSIXPROC set to non-empty definition in order to include POSIX processes emulation
CFG_FS_EXT2 set to non-empty definition in order to include ext2 filesystem implementation
CFG_FS_FAT set to non-empty definition in order to include FAT filesystem implementation
CFG_NETWORK set to non-empty definition in order to include Networking Support Library
CFG_LIBC set to non-empty definition in order to include Standard C Support Library
CFG_PCIHOST set to non-empty definition in order to include PCI host (must be supported by the platfrom)
CFG_EHCI_USBHOST set to non-empty definition in order to include EHCI USB host (must be supported by the platfrom)
CFG_KEYBOARD set to non-empty definition in order to include PC keyboard support
CFG_NIC_DM646x_EMAC set to non-empty definition in order to include DM646x EMAC support (evmdm6467 platform only)
CFG_NIC_PCNET32 set to non-empty definition in order to include DM646x EMAC support (PC and malta platforms)
CFG_UART set to non-empty definition in order to include UART support (device drivers according to platform)
CFG_PL011 set to non-empty definition in order to include PL011 UART support (ARM versatile platform)
CFG_16x50 set to non-empty definition in order to include 16x50 UART support (x86 PC, ARM evmdm6467, MIPS malta platforms)

More options will be added.

Some options may be manipulated via via makefile's definitions. E.g. if you don't need some specific drivers, you may remove relevant object files from DRV_OBJS, if you don't need network support you may set NET_OBJS to blank.

Options configurable in config.h

The main SeptemberOS configuration file is header config.h. Currently it is modified by hand, no auto-configuration tool is provided. You may easily modify build system in order to use one of auto-configuration tools.

The following options are configured in config.h:
Task manager configuration
TICKS_PER_SEC timer ticks per second (configures timer resolution)
TICKS_PER_SLICE timer ticks per execution slice (for time-sharing tasks only)
MAX_TASKS limit of number of concurrently existing tasks
MAX_PRIORITY_LEVELS limit of number of priority levels. Along with MAX_TASKS determines size of tasks table
SYS_PRIORITY_LEVELS number of priority levels reserved for system tasks (like IRQ bottom-halves, TCP helper tasks etc). User tasks should run with lower priority, except for cases when they should deliberately take over system tasks
HIGHEST_USER_PRIORITY_LEVEL highest priority level normally available for user tasks. By default set to SYS_PRIORITY_LEVELS
DEF_PRIORITY_LEVEL default priority level for user tasks. By default is set to (NUM_PRIORITY_LEVELS / 2)
IDLE_PRIORITY_LEVEL priority level for idle task. Idle task is scheduled by the task manager when no other task is runnable. It should have the lowest priority in the system. By default is set to (NUM_PRIORITY_LEVELS - 1)
START_APP_IN_TASK determines whether app_entry() starts already in task context or is just given control when all initialization is complete
INIT_TASK_PRIORITY priority of init task (which runs app_entry)
INIT_TASK_OPTIONS options of init task (which runs app_entry)
DEF_STACK_SIZE default stack size with which new tasks are started
MAX_IRQ_HANDLERS limit of number of shared IRQ handlers per IRQ number. Determines size of interrupt handlers table
IRQ0_BH_LEVEL - IRQnn_BH_LEVEL priority level of IRQnn bottom half handler
MASK_UNHANDLED_INTR determine whether to mask all interrupts that don't have handlers installed, in interrupt controller
I/O configuration
POSIX_IO Determines whether POSIX I/O compatibility layer is included. Determined by CFG_POSIXIO configuration in makefile
MAX_FILES limit number of file structures available for POSIX I/O compatibility layer (determines size of files table and limit of file descriptors)
MAX_DISKS limit number of disk interface structures available for POSIX I/O compatibility layer (determines size of disks table)
MAX_FILESYSTEMS limit number of filesystem interface structures available for POSIX I/O compatibility layer (determines size of filesystems table)
Network configuration
SOCKETS Determines whether sockets API is included. Determined by CFG_NETWORK configuration in makefile
TCPIP Determines whether TCPIP implementation is included. Determined by CFG_NETWORK configuration in makefile
MAX_NET_INTERFACES number of network interfaces. Determines size of table of network interfaces
ETH_TBL_SIZE Number of entries in ethernet addresses hash table
DEF_IP_ADDR Default IP address (static)
DEF_IP_ADDR_STR Default IP address (static) string representation as "x.y.z.a"
MAX_TCP_RETRANSMISSIONS Maximum amount of times the TCP implementation will attempt to retransmit a segment before deciding that connection is broken
DEF_TCP_RETRANSMIT_INTERVAL TCP retransmission timer expiration interval (in system timer ticks)
Misc. configuration
MAX_CMD_PARAMS limit of number of parameters for process emulation (exec() and friends)
Drivers configuration
IDE_NUM_BUSES number of IDE buses to query (for IDE driver)
FIRST_IDE_DISK index of first IDE disk in disks table (IDE disks take consequent places in disks table)
STDIN_DEVNAME Device name of stdin. See Configuring Device Drivers
STDOUT_DEVNAME Device name of stdout. See Configuring Device Drivers
STDERR_DEVNAME Device name of stderr. See Configuring Device Drivers

Configuring Device Drivers

Device drivers are configured statically in a table in config.h. Every SeptemberOS device driver exports a structure of type drv_entry. This structure includes driver's entry points and other information important for OS.  When configuring a driver it is necessary to include three steps:

Configuring Device Names

Drivers may be assigned convenient names, such as "/dev/tty", "/dev/serial", etc. While this is not necessary in order to make the driver fully functional and accessible, this is necessary in order to make the driver accessible via POSIX compatibility layer (open(), read(), write() etc.) Devices names are configured in dev_tbl table. There are no limitations on device names, they may be any ASCII\0 string. Care should be taken to avoid duplicating device names in filesystem files, as device table is always accessed first by POSIX compatibility module and filesystem entity that shares name with a device will never be accessed.

Configuring System Tasks

System tasks are IRQ bottom-halves and other high-priority tasks that normally run in priority range 0 - SYS_PRIORITY_LEVELS. Their configuration should rarely have reason to be changed; new system tasks may need to be configured when new drivers are added, and they have IRQ bottom halves. In order to configure a system task to run at start-up, simply add its entry point and priority to sys_tasks table. Note that the same mechanism may be used to start automatically user tasks (with priority in user range) if for some reason deliberately starting tasks from app_entry() is not desirable.


Writing SeptemberOS Applications

SeptemberOS beta version 1.0 doesn't include and IDE. You may use any text editor in order to compose your application's source files, then add them to makefile similar to sample applications for your platform, as described above. There are no limitations on what your application should be or how complex it can be; below are some considerations for programming SeptemberOS application.

Application vs Kernel

Usually when considering applications versus OS kernel we consider different level of privileges. In typical desktop OS (and in many embedded OS) applications run in so-called user (unprivileged) mode, while OS kernel runs in kernel (or system, privileged) mode. Kernel then has access to all system and hardware resources, while applications have access only to resources allowed them by the kernel. This way the system is protected from erroneous or malicious access by applications, thing that is invaluable when a lot of third-party code from unknown source runs in the system.

However, in an embedded system advantages of such protection are not so obvious, while drawbacks are felt. SeptemberOS follows philosophy of "classical" embedde RTOS that such protection is not necessary. SeptemberOS doesn't provide immediate means to load programs dynamically at all (although such means may be developed and implemented by user application), therefore there is usually no third-party code running on the system. EmbeddedOS philosophy tells that the OS is more of a convenience library and useful building blocks; it is not a kind of exclusive system manager and guard.

Under SeptemberOS applications run on the same level as OS kernel. All system and hardware resources are available to applications. Therefore, while it is convenient and conceptually easier to work with drivers configured separately (as shown above), complete drivers functionality may be included in application code. Application may interfere with and alter task management, may install its own interruts handlers etc.

All system API available for the kernel is also available for applications. SeptemberOS doesn't enforce a programming model, but we maintain that the OS framework is designed to be useful and a good reason must exist for changing or interfering with it.

Processes and Threads

SeptemberOS doesn't support processes in traditional concept. The OS doesn't provide address space separation, and memory model is like single-process multi-threaded. POSIX compatibility layer provides processes emulation via exec() (and friends) API calls. The calls are convenience only, they are actually front-ends to tasks creation calls. fork() is not implemented due to incompatibility with SeptemberOS'es execution model.

At the same time, SeptemberOS execution model is completely compatible with traditional threads. POSIX compatibility layer provides pthreads front-ends for most task creation, termination and priority management needs.

SeptemberOS application can run in multi-tasking framework, taking full advantage of the OS'es real-time task scheduler. (See Task Manager). Alternatively, many embedded applications are implemented in form of one main loop, which handles events and does different processing. For such applications it is possible to leave out of build the Task Manager, and use other SeptemberOS services (such as POSIX I/O , standard C library, devices support, timers, interrupts etc).

POSIX I/O

SeptemberOS provides POSIX I/O interface via open(), read(), write(), ioctl(), dup() and other (see POSIX Support Library). The following POSIX I/O facilities are available:

Standard C Library

SeptemberOS provides with convenience of standard C library. Most functions declared in stdio.h, string.h, stdlib.h, time.h and others are implemented. The most notable exception is math.h functions, that are not implemented in the current version (see Standard C Support Library).

Networking

SeptemberOS includes support for IPv4 TCP/IP over Ethernet, interfaced via Berkeley Sockets API.

Porting Considerations

SeptemberOS is strongly oriented on providing industry-standard interfaces, in order to make porting of existing code easy. Porting of applications from POSIX-compliant OS, which uses C standard library is near-trivial. Things to remember when porting are:
Porting of OS to another platform is also almost straight forward: you should implement Architecture Support Library for your platform, according to its specification and provide the necessary OS entry points. You may take as a reference one of existing implementations and just rewrite specifics of your platform.

Timing and Delaying

In many cases occasions embedded code must be timed and / or delayed. SeptemberOS provides the following means for timing / delaying:
When using udelay() it is important to properly configure it for every platform. udelay() macro is implemented with two static counters in file sosdef.h. Busy-polling uses two constant static variables which are pre-calibrated for every platform. When configuring SeptemberOS for a new platform it is important to calibrate those values once and then to define them in sosdef.h

SeptemberOS provides calibration code in file devman.c, which will calibrate those values during initialization of Device Manager component if CALIBRATE_UDELAY was defined in config.h. Once the calibration code executed, it will output to serial port the following:

"init_devman(): calibrated udelay() -- before calculation: c1=nnnn, c2=mmmm"
"init_devman(): calibrated udelay(): c1=nnnn, c2=mmmm"

Then define in sosdef.h under your platfrom #ifdef: CALIBRATED_UDELAY_COUNT1 to c1 value printed in the second line (nnnn) and CALIBRATED_UDELAY_COUNT2 to c2 value printed in the second line (mmmm). Ignore the first line (with "-- before calculation").

Calibration causes a small delay in system start-up, and it is not necessary once the values are calibrated. After you have c1 and c2 values, undefine CALIBRATE_UDELAY in config.h

You may run any application built with CALIBRATE_UDELAY defined for calibration purpose. For simplicity we recommend running uart-sample provided for every supported platform.

Note that you have to re-calibrate udelay() when changing CPU clock even on the same platform on which you already did that.

Debugging

SeptemberOS evaluation release doesn't include a dedicated debugger. Several means, however, may be used to test and examine execution of your code:

Working with QEMU System Emulator

Several SeptemberOS targets may run under emulation platform. Working with emulator is a great asset during development and debugging due to its very fast setup and load time, no platform electrical and mechanical malfunction and debugging support.

The following QEMU targets are supported by SeptemberOS:

ARM versatile qemu-system-arm -M versatilepb -kernel image.elf -m 128 -nographic
MIPS malta qemu-system-mips -M malta -kernel image.elf -m 128 -nographic
x86 pc qemu -L "c:\program files\qemu\pc-bios" -fda boot.flp -hda septos.qcow2 -boot a -net nic,model=pcnet,macaddr=11:22:33:0B:0C:0D -net tap,ifname=lan2 -serial file :uart-output.txt -localtime

SeptemberOS was tested with QEMU system emulator on all released platforms except for evmdm6467 (for which QEMU emulation support doesn't exist as of this writing).

QEMU includes a GDB stub, which makes for a very convenient debugging environment. When used with "-s" switch QEMU starts with GDB stub listening on TCP port 1234 (you may use "-gdb" option to specify a different port), and when applied with "-S" option, QEMU freezes after loading an image until you input "c" command in its monitor or command it to run with a debugger.

QEMU allows you convenience of rapid development and debugging on a single host. In order to run a debugging session in such a simple way you may use the following steps:
Current directory is assumed to be "src/" directory in SeptemberOS distribution. (You will see a sample file septos.gdb that contains the commands in that directory). Use cross-platform GDB for your target, you may use "gdb -x" if you keep your connection commands in command file.

Please refer to QEMU website for documentation on QEMU operations and invocation.

SeptemberOS is also tested and provided as VmWare virtual machine.


Device Drivers

SeptemberOS evaluation release is supplied with the following device drivers.

Device Source Directory (under drivers/) Platforms
PC keyboard keyboard/ x86 (pc)
IDE disk bus/ide/ x86 (pc)
PCI host bus/pci/ x86 (pc)
USB host (EHCI) bus/usb/ x86 (pc)
I2C master bus/i2c/ arm (evmdm6467)
i8255x NIC net/ x86 (pc)
pcnet32 NIC net/ x86 (pc), mips (malta)
DM646x EMAC NIC net/ arm (evmdm6467)
16x50-compatible UART serial/ x86 (pc), arm (evmdm6467), mips (malta)
PL011 UART serial/ arm (versatile)
text-mode VGA terminal terminal/ x86 (pc)

Some additional devices (system timer, real-time clock, system interrupt controller) are supported via system calls rather than via device drivers API.


Sample Applications

SeptemberOS evaluation release is supplied with the following sample applications:


Sample Source Directory (under samples/) Description Platforms
hello hello/ "hello, world" sample. Demonstrates basic SeptemberOS simple program without tasks, build and integration all
sample sample/ basic multitasking sample (PC only) x86(pc)
uart uart/ UART simple echo sample. Shows how to work with UART device driver directly (bi-directional)
uart-sample uart-sample/ basic multitasking sample via UART (all platforms). Used as testbench for new arch/mach ports (basic taskman, memman, interrupts, timer, uart support) all
monitor monitor/ command monitor application, works with keyboard and display terminal devices: devices API, filesystems and POSIX I/O testing (open, read, write, rename, delete, stat), system information (mem, ip), simple TCP ping-pong server. the monitor has three front-ends: terminal (term-mon, PC only), UART (uart-mon, all platforms) and telnet (telnet-mon, all platforms) all
tcp-srv network/ simple TCP server, may be used to serve telnet client. Basic networking, shows ease of porting sockets application all
tcp-srv-mt network/ multi-threading (multi-tasking) version of simple TCP server, opens a new pthread for every connection. Sample server (sockets), pthreads API all
telnet-srv network/ TELNET protocol implementation (server). Opens a new task (thread) for every connection. Sample server (sockets), pthreads all
tcp-client network/ TCP client sample for testing sockets API all
http-srv network/ basic HTTP server. Opens a new task (thread) for every connection. Sample server (sockets), pthreads, file I/O (tests libc, POSIX I/O, filesystem, TCP/IP) all
socktests network/ thorough test suite for sockets API. Tests UDP chat (bi-directional), TCP server, TCP client, select() multiplexing on two
sockets, non-blocking accept() and non-blocking UDP chat (bi-directional) 		all


Version 1.0 beta Release Statement

Source Code

SeptemberOS version 1.0 beta is released in form of buildable source code. The following architectures and platforms are supported:


Architecture Platform Supports Status Tested
ARM9 Versatile (ARM) Dual timers, vectored and secondary interrupt controller, PL011 UART Experimental QEMU system emulator
ARM9 EVMDM6467 (Spectrum Digital) System timer, interrupt controller, power and sleep controller, UART, I2C controller, EMAC/PHY Stable EVMDM6467 module
x86 PC System timer, interrupt controller, UART, PCI host, ehternet controllers PCNET32 and 8255x, RTC, USB host (experimental), IDE host, keyboard, VGA monitor (text mode) Stable PC, QEMU system emulator, VmWare system emulator
MIPS 24Kf Malta System timer, interrupt controller, UART, RTC Experimental QEMU system emulator

x86 port serves as "master port", all applicable features are tested first on it and then ported to other platforms (with exception of platform specific features). Before porting, features are tested on QEMU, VmWare and x86 32-bit computer.

The following filesystems are supported:
ext2 is more tested and mature development. It is used as reference filesystem implementation in SeptemberOS.

Note: there is a legal issue with long file names suppor on FAT filesystem currently. Microsoft, the developer of FAT and holder of several patents on it, is enforcing patents related to long file names support. Following, SeptemberOS doesn't support long file names in its FAT implementation; only standard FAT12 and FAT16 structures are supported. Currently SeptemberOS implementation skips any occurances of long file name entries and works only with their short counterparts.

Build and Debug Environment

The following build environment was tested and used for daily builds:

Run-time Environment

The following testing environments were used:

Known Issues List

Component Issue Platform Status
FAT Long file names are not supported, but related in directory search All Decision pending
RTC services not implemented versatile, evmdm6467 Development
Standard Library math functions not implemented all Definition
Network Library IP fragmentation mechanisms are not functional all Resolution pending


Current Development

The following development is currently in progress:

Component Feature Platform Status
Video drivers Support for video acquisition and output evmdm6467 Development
NIC drivers Support for SMSC91C111 ethernet card versatile Non-stable, testing
USB host Generic USB host interface all Definition
RTC Support for real-time clock versatile, evmdm6467 Development
PCI host Support for PCI host malta Development
Build environment Support for Windows-hosted build environment all Definition


SeptemberOS Version 1.0 beta Evaluation License

0. This license defines terms and conditions for use of SeptemberOS embdedded RTOS. The term "SeptemberOS" refers to full original package or any sub-package derived from it. SeptemberOS is copyright (c) Daniel Drubin, 2007-2010. All rights reserved. You (either an individual or a single entity) are not required to sign or somehow acknowledge your use of SeptemberOS. However your use of SeptemberOS must fully comply to this license; any misuse automatically terminates this license and revokes from you any right to further use SeptemberOS. You have no other rights on SeptemberOS except for non-exclusive rights listed here.

1. COPYING. You may make an unlimited number of copies, modified or unmodified of SeptemberOS for your own personal use (for example, back-up). You may make any number of modified copies for your own use.

2. DISTRIBUTION. Any distribution of SeptemberOS and derived work is not allowed. This includes the OS code, complete or partial, in source code or compiled form and all its interfaces. In particular you may not distribute software and hardware designed to interface with SeptemberOS in any way and patches to SeptemberOS.

3. TERMS OF USE. This version of SeptemberOS may be used free of charge for any evaluation purpose that doesn't include distribution of it or derived work. If you intend to use SeptemberOS for production consider applying for a separate license. This license is not time limited; you may use this version of SeptemberOS as long as you wish. It does not grant you any rights on the forthcomming versions of SeptemberOS and doesn't garantee future releases of SeptemberOS licensed under the terms of this license.

4. LIMITATION OF LIABILITY. In no event, except for when it is explicitly stated by the applicable law, shall Daniel Drubin be liable for any special, incidental, indirect, or consequential damages (including but not limited to profit loss, business interruption, loss of business information, or any other pecuniary loss) arising out of the use of or inability to use SeptemberOS, even if he has been advised of the possibility of such damages.


-----
All trademarks, registered trademarks and reserved words belong to their respective owners.

References

[1] The core of the Single UNIX Specification. IEEE Std 1003.1,2004 Edition (POSIX-1.2001). http://www.unix.org/version3/ieee_std.html

[2] The ISO C Language Standard. Programming Languages – C (ISO/IEC 9899:1999). http://www.open-std.org/JTC1/SC22/WG14/www/standards

[3] OSI Networking Model. Information Technology – Open System Interconnection (ISO/IEC 7498-1). http://standards.iso.org/ittf/PubliclyAvailableStandards/s020269_ISO_IEC_7498-1_1994(E).zip

[4] Ethernet Address Resolution Protocol. RFC 826. http://www.faqs.org/rfcs/rfc826.html

[5] Internet Protocol. RFC 791. http://www.faqs.org/rfcs/rfc791.html

[6] Transmission Control Protocol. RFC 793. http://www.faqs.org/rfcs/rfc793.html

[7] User Datagram Protocol. RFC 768. http://www.faqs.org/rfcs/rfc768.html

[8] Dynamic Host Configuration Protocol. RFC 2131. http://www.faqs.org/rfcs/rfc2131.html

[9] Internet Control Message Protocol. RFC 792. http://www.faqs.org/rfcs/rfc792.html

[10] Sourcery G++ Lite Edition. CodeSourcery's free, unsupported command-line version of Sourcery G++ sponsored by its hardware partners. http://www.codesourcery.com/sgpp/lite_edition.html

[11] QEMU, generic and open source machine emulator and virtualizer. About - QEMU http://wiki.qemu.org/Main_Page

[12] VmWare Download. VmWare. http://www.vmware.com/products/player/