INOS
Kernel

Overview

INOS bases on a fully preemptive multicore kernel with 32 priorities. The number of tasks it can handle is only limited by the available target memory. Task switch times are independent of the number of tasks currently running in the system. The system behaviour is therefore fully deterministic in all situations.

The kernel and the application are written in C++ and compiled with GNU Compiler Collection (GCC). Currently with gcc 4.7.4.


Booting


Singlecore

To get a defined booting sequence, INOS provides a so called StartFunction mechanism. It bases on a macro which allows to define named global functions, which are called at boot time in alphabetical order. Example:

{
// do some initialisation stuff here
//
// ...
}
Definition cinosmcmodule.h:1900

The function name in this example is _INI_1000_MyInit. As mentioned above, all functions are called in alphabetical order. Any naming could therefore be used. Due to the fact, that also INOS itself is using this mechanism to boot itself, there is a predefined naming schema, which has to be used to get a defined behaviour. It looks like this :

The _INI_nnnn part is mandatory and defines the sequence. Numbers from 0000 til 0999 are reserved by INOS and should not be used. Numbers 1000 til 9999 are reserved for the application.

Hint: To get an overview over all defined start functions in the system (including INOS ones), you can press shift-ctrl-T in iDev which opens a global element search dialog. In the Choose an element field just write _INI_. After a few seconds you get a list of all startfunctions in the Matching elements part.


Multicore

Multicore systems are booted like single core ones, but one core after the other, or in other words first all start functions of core 0, then all of core 1, ... . To be able to tell the system which start functions need to be called by which core, we introduced an extended macro called StartFunctionEx, which allows to additionaly define a coremask. Example :

{
// what core are we running on ?
if (GetCoreId() == 0) {
// do some core 0 initialisation stuff here
//
// ...
}
else {
// do some core 3 initialisation stuff here
//
// ...
}
}

The coremask in the example is 0x9 (bit 0 and bit 3 are set to 1), which means, that this function is called by core 0 and core 3 on e.g. a GIN-SAM4 Quad. The core mask has the following meaning :

Hint: To have multiple cores available on a multicore target (currently COP-MAS2, IMP-MAS4, GIN-PCIe and GIN-SAM4), you have to activate Enable multicore support in your iDev project.


Task

To get your application running, you always need to create at least one task. A task aka thread has always one of three states :

  • running task is currently running, per core always just one task can be in this state
  • ready task is ready to run, but doesn't get cpu power due to priority, scheduling reasons
  • suspended task doesn't require cpu power, it stays in this state until it is resumed (put into ready state) by an other task

A task can have the following different suspended states

  • 1 pure no special reason, e.g. after creation
  • 2 debug a debugger has stopped the task
  • 3 sleep task is waiting on a Sleep command
  • 4 trap task trap occured (e.g. data access exception, division by zero, ..)
  • 5 mutex task is waiting on a mutex
  • 6 sync task is waiting for a sync object to be signaled
  • 7 assert assertion occured
  • 8 exception task got an exception without a valid try/catch frame
  • 9 beyond action task returned from virtual Action method and stays in the main loop
  • 15 zombie task was killed but not yet destroyed

Hint: After creation of a task, it is in the state 1 (pure suspended) and needs to be resumed to get it running.


Creation

There are two possibilities to create an own task. You can either tell the system which function to use as your main task loop or create an own class derived from CINOSTask and overwrite the virtual method CINOSTask::Action().

See also CINOSTask::CINOSTask

Example with simple task loop :

#include <inos.h>
void MyTask()
{
// stay here forever
while (1) {
// ... rrr ...
Sleep(100);
}
}
{
// create task with lowest priority
true, defDefaultTimeSlice, false, (void*) &MyTask);
// start it
Resume(task);
}
Definition cinostask.h:52
#define DF_INOS_TASK_PRIO_LOWEST
Definition inosdefine.h:187
#include <inos.h>
void MyTask(void* apParam)
{
// do some work with megaParams
// ...
Exit(0);
}
// ...
// start it
Resume(&cool);
// wait till completed
uint32 uExitCode = cool.Join();
// ...

Example with own class :

#include <inos.h>
class CMyTask : public CINOSTask
{
// publib members
public:
// constructor
CMyTask(char* apName)
{};
// protected members
protected:
// main task loop
virtual void Action()
{
// stay here forever
while (1) {
// ... rrr ...
Sleep(100);
}
};
};
{
// create task with lowest priority
CINOSTask* task = new CMyTask("Cool");
// start it
Resume(task);
}

A task is always running on the on core it was created with.

Example with multiple tasks on multiple cores :

#include <inos.h>
void MyTask1()
{
// stay here forever
while (1) {
// ... rrr ...
Sleep(100);
}
}
void MyTask2()
{
// stay here forever
while (1) {
// ... rrr ...
Sleep(100);
}
}
{
// check core id
switch(GetCoreId()) {
case 0 : Resume(new CINOSTask("Cool0", defDefaultStackSize, DF_INOS_TASK_PRIO_LOWEST, true, defDefaultTimeSlice, false, (void*) &MyTask1)); break;
case 1 : Resume(new CINOSTask("Cool1", defDefaultStackSize, DF_INOS_TASK_PRIO_LOWEST, true, defDefaultTimeSlice, false, (void*) &MyTask1)); break;
case 2 : Resume(new CINOSTask("Cool2", defDefaultStackSize, DF_INOS_TASK_PRIO_LOWEST, true, defDefaultTimeSlice, false, (void*) &MyTask2)); break;
case 3 : Resume(new CINOSTask("Cool3", defDefaultStackSize, DF_INOS_TASK_PRIO_LOWEST, true, defDefaultTimeSlice, false, (void*) &MyTask2)); break;
}
}

This example (running on a GIN-SAM4 Quad) creates on core 0 and core 1 a task running whithin MyTask1 and on core 2 and core 3 a task running whithin MyTask2. Due to the fact that INOS has an own scheduler per core, it would be possible to use the same name for all tasks (e.g. only 'Cool'), but we do NOT recommend that.

A task can live switch to an other core using the function SwitchCoreish(Core).

Example :

// ...
// assume we're running on any core and want to switch to core 3
SwitchCore(3);
// now running now on core 3
// ...

Hint: To debug a task running on a core different from 0, you need to change your target to MyTarget\@n where n defines the required core number, e.g. MyTarget\@2 for core 2.


Priority

The importance of a task is defined by its priority. Task priorities are numerical values from 0 to 31. The lower the value, the higher its priority. Priority 0 is therefore the highest possible one. INOS always gives cpu power to the task with highest priority which is currently in state ready. Tasks with the same priority are given cpu power in the sequence they were put into ready state.

A task only gets cpu power if its the one with currently highest priority of all ready tasks in the system. Assume the following situation: One or multiple tasks with a given priority always stay in state ready. This would lead to starvation of all other ready tasks with lower priority. There are different posibilities to avoid such situations. Tasks with a high priority need to suspend themself to allow lower priority tasks to get cpu power. Task which need to run continuously must use a low or better the lowest priority.

Hint: If you assign a priority different from lowest to your task, you need to be very careful. It could lead to starvation of other tasks or even to a non functional system.

INOS defines different priority groups.

  • 0..15 realtime reserved for realtime tasks, e.g. fieldbus handlers and interrupt tasklets
  • 16..23 communication reserved for communication tasks, e.g. INCO communication, inter core communication, ...
  • 24..30 user reserved for user application
  • 31 background priority, e.g. used by the Lua garbadge collecter

Hint: You should only work with user task priorities unless you really know what you are doing.

Hint: On GIN-SAM3 and INFO-PCIe, one has to select the iDev feature INOS Kernel Type = standard to get 32 priorities.

INOS works with fixed priorites or in other words, it does not implement dynamic priority changes to avoid priority inversion situations. Priority inversion is defined by the following situation. Assume you have a low and a high priority task which need to access the same object and mutual exclusion is done with a mutex. If the low priority task owns the mutex, the high priority task is suspended till the mutex gets released. Or in other words: allthoug the task has a high priority, it gets blocked by a lower prio task.

Hint: Avoid priority inversions by accessing sync objects (e.g. mutexes) from tasks with different priorites, as this could lead to a non functional system or at least to fieldbus task overruns. You could avoid such situations by using INOSDisableInterrupts and INOSEnableInterrupts at least if the two tasks are running on the same core. But the mentioned two functions need to be used with care. Ask Indel if your not sure!.


Preemption

Preemption occures if a currently running task is interrupted due to a state change from suspended to ready of a higher priority task, or if the timeslice of the actual running task has expired. Examples :

  • a system interrupt occured, which resumes a high priority interrupt tasklet (e.g. fieldbus interrupt)
  • a running task uses the function ChangePriority() to lower its priority
  • a running task puts a message into a queue on which a higher priority task is waiting on
  • a running task signals a sync object where a higher task is waiting on
  • a running task releases a mutex, where a higher task is waiting for

Relinquish

This mechanism allows to share the cpu power with multiple tasks of the same priority. If a task calls Relinquish(), all other tasks with the same priority get cpu power before the caller gets cpu power again. This is a more efficient way to share cpu power with other tasks, than to rely on the timeslicing mechanism. Note that by calling this function, the calling tasks time slice will be reset. One implication of that is that by calling this function, one can assure that when getting back CPU power (and probably it will never be taken away from the calling task, if there no other tasks with the same priority request cpu power), this task will have the full time slice available till it'll be preemted.

Example of two tasks sharing cpu power with Relinquish :

#include <inos.h>
void MyWorker()
{
// stay here forever
while (1) {
// lets do something here
// ...
Relinquish();
}
}
{
Resume(new CINOSTask("Worker1", defDefaultStackSize, DF_INOS_TASK_PRIO_LOWEST, true, defDefaultTimeSlice, false, (void*) &MyWorker);
Resume(new CINOSTask("Worker2", defDefaultStackSize, DF_INOS_TASK_PRIO_LOWEST, true, defDefaultTimeSlice, false, (void*) &MyWorker);
}

Hint: You can visualise the behaviour with the Indel Tasklog tool.


Timeslice

An other mechanism to share the cpu power with multiple tasks of the same priority is the timeslicing mechanism of INOS. A task is preempted by the system if it didn't release cpu power with e.g. Relinquish() or Suspend() for the defined timeslice. The system then preempts the task and gives cpu power to all other task with the same priority before the preempted task gets cpu power again.

Example of two tasks sharing cpu power with time slicing mechanism :

#include <inos.h>
void MyWorker()
{
// stay here forever
while (1) {
// lets do something here
// ...
}
}
{
Resume(new CINOSTask("Worker1", defDefaultStackSize, DF_INOS_TASK_PRIO_LOWEST, true, 100, false, (void*) &MyWorker);
Resume(new CINOSTask("Worker2", defDefaultStackSize, DF_INOS_TASK_PRIO_LOWEST, true, 200, false, (void*) &MyWorker);
}

In the example Worker1 is preempted after 100us and Worker2 after 200us. This is also a possibility to give tasks a pseudo priority or more cpu power.

Hint: You can visualise the behaviour with the Indel Tasklog tool.


Determinism

Task switching time is constant. It is independent of the number of tasks currently in state ready. It is also well defined when a task gets cpu power. Higher priority based tasks are running before lower prior tasks. Tasks with the same priority are executed in the sequence they were put into ready state.


Hook

An other possibility to get CPU power for the application is to work with hooks. A hook is a function or class method witch is called in the context of a cyclic running fieldbus task. A hook is registred at the corresponding fieldbus instance. There are two main parameters, the requested cyceltime and the cycle id. The cycle id is a uint16 value with the following meanings

15 10 9 8 7 0
--------------------------------
| | | |
--------------------------------
| | |
| | cycle number (0..255)
| category (0..3)
core id (0..63)

There are several macros available to handle the cycleid value, see INOS_CYCLEID.

Example registering a 4kHz hook at GinLink category 2 :

#include <inos.h>
#include <cinosbus.h>
void MyHook()
{
// called in the context of GINLink2 with 4kHz
//
// do some work here
//
}
{
// get pointer to ginlink
CINOSBus* ginlink = CINOSBus::FindBus("GinLink");
if (0 != ginlink) {
// locals
uintid uHookId;
// register hook with 250us, category 2, cycle number don't care)
uint32 uResult = ginlink->RegisterHook(uHookId, (void*) &MyHook, 0, 250000, INOS_CYCLEID(0, 2, 0xff), DF_INOS_BUS_HOOK_ORDER_DEFAULT);
if (INOS_OK != uResult) {
// hook registration failed for some reason
//
// ...
}
}
}
Short comment.
#define DF_INOS_BUS_HOOK_ORDER_DEFAULT
bus hook order 'default'
Definition cinosbus.h:137
#define INOS_CYCLEID(auCoreId, auCategory, auCycleNumber)
Create cycle id from core id, category and cycle number. A cycle id is a uint16 value,...
Definition cinosbus.h:165
Definition cinosbus.h:600
uint32 INOS_OK
Definition inoserror.h:1677

One has to be aware of the fact, that the function MyHook in the example is called in the context of the high priority realtime task GINLink2. This task is also responsible for all the process image stuff as well as axes controlling. One should therefore only code things that are really necessary to be done cyclic. Using too much time in a hook results in possible fieldbus overruns and ends up in a non functional system.

Hint: You should only work with breakpoints in a realtime hook if you either really know what you are doing or if you work without real fieldbus hardware and therefore fully simulated. To measure the time needed for code parts, use testpoints instead (in iDev : Breakpoint Types -> INOS Testpoint)

To reduce the just mentioned risk, one has the possibility to register a so called post hook. It has pretty much the same behaviour but is called in the context of a lower priority task that is just resumed after the corresponding main cyclic task has done it's work.

Example registering a 4kHz post hook at GinLink category 2 :

#include <inos.h>
#include <cinosbus.h>
void MyHook()
{
// called in the context of GINLinkPost2 with 4kHz
//
// do some work here
//
}
{
// get pointer to ginlink
CINOSBus* ginlink = CINOSBus::FindBus("GinLink");
if (0 != ginlink) {
// locals
uintid uHookId;
// register hook with 250us, category 2, cycle number don't care)
uint32 uResult = ginlink->RegisterPostHook(uHookId, (void*) &MyHook, 0, 250000, INOS_CYCLEID(0, 2, 0xff), DF_INOS_BUS_HOOK_ORDER_DEFAULT);
if (INOS_OK != uResult) {
// hook registration failed for some reason
//
// ...
}
}
}

The advantage of working with a post hook is that one can work with breakpoints in its hook without disturbing the system.

Hint: If you set a breakpoint in MyHook, you'll see that the task GINLinkPost2 stops at the required point.


Syncing mechanisms


Critical sections

A critical section is a code part where multiple tasks or even cores need to access a shared resource. One way to define a critical section is to work with a mutex. With a mutex you can ensure that always only one task is running through a code snippet, independent of the core the task is running on.

Example :

#include <inos.h>
uint32 uCounter=0;
void IncCounter()
{
// enter critical section
mutex.Request();
// inc counter
// exit critical section
mutex.Release();
}
Definition cinosmutex.h:36

The example has the small disadvantage, that one has to ensure, that every return path releases the mutex, otherwise the mutex would be locked forever. To make the code even simpler, one can work with the CINOSLock class.

Example :

#include <inos.h>
uint32 uCounter=0;
void IncCounter()
{
// enter critical section
// inc counter
}
Definition cinosmutex.h:167

The mutex is requested in the CINOSLock constructor and released in its desctructor.

An other way to define a critical section is to work with INOSDisableInterrupts and INOSEnableInterrupts.

Example :

#include <inos.h>
uint32 uCounter=0;
void IncCounter()
{
// enter critical section
// inc counter
// exit critical section
}

This is the most efficient but also the most dangerous way. If you disable interrupts, you need to be aware of the fact, that everything is blocked till you reenable them again! Disabling interrupts only works on one core. If you need to define critical sections for multiple cores, you have to work with mutexes.

Use INOSDisableInterrupts and INOSEnableInterrupts if :

  • only tasks running on one core need to be handled
  • the section to be secured is just a short code part (some assembler instructions)
  • you have tasks with different priorities accessing the resource, see also priority inversion, Priority

Use mutexes if :

  • the resources to be secured is accessed by multiple cores
  • all tasks accessing the resource have the same priority
  • if you don't know how long the code part to be secured is

Hint: Ask Indel if you're not sure how to handle a critical section issue.


Sync object

Sync objects are used to interact between tasks even running on different cores without having to actively poll for a corresponding variable for example. They are represented by the class CINOSSync. The INOS motion library uses sync objects to handle synchronous and asynchronous library calls.

Example :

void LetsMove()
{
// sync object
// pointer to axis
CINOSPhysicalAxis* x = CINOSPhysicalAxis::GetAxis("X");
// move to 100
x->Move(100.0, &sync);
// wait till done
sync.Wait();
}
Provides physical aka real axis functionality.
Definition cinosphysicalaxis.h:286
Definition inos_syn.h:67

All functions or class methods accepting a pointer to a sync object also accept the two special values

#define DF_INOS_ASYNCHRONOUS
Definition inosmacro.h:337
#define DF_INOS_SYNCHRONOUS
Definition inosmacro.h:332

Example :

void LetsMove()
{
// pointer to axis
CINOSPhysicalAxis* x = CINOSPhysicalAxis::GetAxis("X");
// move to 100 synchronous (returns at end of move)
x->Move(100.0, DF_INOS_SYNCHRONOUS);
// move back to 0 asynchronus (returns immediately)
x->Move(0.0, DF_INOS_ASYNCHRONOUS);
}

IMPORTANT: If you provide a sync object pointer to a function, always ensure that the sync object stays valid, till it is signaled.

Example (this leads to unspecified bahaviour):

void LetsMove()
{
// sync object
// pointer to axis
CINOSPhysicalAxis* x = CINOSPhysicalAxis::GetAxis("X");
// move to 100
x->Move(100.0, &sync);
// !!! due to the fact, that sync is part of the stack,
// it becomes invalid when we leave the function !!!
}

The example above ends up in unspecified behaviour, due to the fact that the motion library will signal a sync object that is NOT valid anymore.


Semaphore

Semaphores are used to control the access to common resources by multiple tasks. They are represented by the class CINOSSemaphore. a typical use case is the producer consumer problem, see https://en.wikipedia.org/wiki/Producer%E2%80%93consumer_problem

#include <inos.h>
#include <inos_lib.h>
uint32 uProducedCount = 0;
uint32 uConsumedCount = 0;
const uint32 uBufferSize = 10;
void producer()
{
while (1) {
// produce item
// down empty count
empty->Request();
// put item to buffer
// ...
// up fill count
fill->Release();
}
}
void consumer()
{
while (1) {
// down fill count
fill->Request();
// remove item from buffer
// up empty count
empty->Release();
}
}
{
// create semaphores
fill = new CINOSSemaphore("fill", 0, uBufferSize);
// create tasks with lowest priority
true, defDefaultNoTimeSlice, false, (void*) &producer));
true, defDefaultNoTimeSlice, false, (void*) &consumer));
}
Definition inos_syn.h:456

The above example shows a possible implementation of the producer consumer problem with CINOSSemaphore with a buffer size of 10.


Queue

Queues implement besides sync objects the most important syncing mechanism in INOS. They are the base of the active object approach of the McRobot framework and are used to build FIFO (first in first out) data channels between tasks.

Example of a simplest active object :

#include <inos.h>
class CMyModule : public CINOSTask
{
// publib members
public:
// constructor
{
// create queue with 10 entries
m_pQueue = new CINOSQueue("Commands", 10, sizeof(uint32));
};
// module command
void Command1()
{
uint32 uCmd = eCmdCommand1;
// put command to queue
m_pQueue->Put(&uCmd);
}
// protected methods
protected:
// main task loop
virtual void Action()
{
// stay here forever
while (1) {
// wait for command
uint32 uCommand = 0;
// check command
switch(uCommand) {
}
}
};
// internal module command
void iCommand1()
{
// do some work here
//
// ...
}
// protected members
protected:
// commands
enum {
};
// object queue
};
{
// create task with lowest priority
CMyModule* task = new CMyModule("MyModule");
// start it
Resume(task);
// emit command
task->Command1();
}
Definition inos_syn.h:513

The example creates an active object which provides commands. The idea of an active object is to implement a software module with a defined interface and an own task. All commands are handled in the context of the modules task. The big advantage of this approach is, that no critical sections are needed due to the fact, that only one task is handling the module's resources. This massively simplifies also debugging. In the above example the method Command1() is called in the context of the caller task. It only writes the required command into the modules command queue. The command itself is implemented in the internal protected method iCommand1().


Memory management

In a realtime operating system it is important to have a deterministic memory management and to avoid memory fragmentation. To solve these issues, INOS provides an own partition based memory managment, implemented by CINOSPartitionMemory. For a seamless integration with arbitrary C/C++ code, INOS has implemented mechanisms so that any call to malloc, free, operator new/delete & friend will actually use INOS heap management.

Every allocation is of a specific size. INOS first rounds that size up to a certain supported values, so called chunk sizes. E.g. allocations between 1..8 Bytes may all fall into the '8 Byte chunk', sizes of 33..40 may fall into the '40 Byte chunk', etc. For every such memory chunk size that is used by either the application or INOS itself, an instance of CINOSPartitionMemory is created which is responsible for this allocation size. Internally all memory blocks of a corresponding chunk size are connected in a linked list. Whenever the application or INOS allocates a memory chunk, the class simply removes the head chunk of the list and returns a pointer to it. If a block is freed, it is added to the head of the linked list again. If there are no more free blocks in the list, the memory is allocated directly from the heap. Memory blocks therefore belong always to the same partition pool and will never be returned to the main heap.

Although global operator new/delete and malloc/free can be used, there's a better way of handling heap if you implement your own struct or class by using the macros DECLARE_DYNAMIC in the declaration part and IMPLEMENT_DYNAMIC in the implementation part. Technically, these macros add class specific 'operator new' & 'operator delete' The benefits of using these macros are many fold:

  • Slightly faster operator new/delete than with the global ones
  • More sanity checks during allocation and freeing (e.g. detects when wrong pointer casting happened during operator delete)
  • Greatly simplifies memory leak detection: Each class, using these macros, is registered in the INCO tree (Target.Memory.Pools.Heap.Partitions among others showing the allocation count and therefore makes memory leaks visible in the INCO tree.

Example :

Declaration part (e.g. myclass.h)

#include <inos.h>
class MyClass
{
// some declarations
//
}
#define DECLARE_DYNAMIC(aClass)
Definition cinospartitionmemory.h:328

Implementation part (e.g. myclass.cpp)

#include <inos.h>
#define IMPLEMENT_DYNAMIC(aClass)
Definition cinospartitionmemory.h:361

These two macros simply overwrite the new and delete operators and implement them with the described partition pool allocator.

Alloc/Free & Real-time tasks :

INOS heap management is quite low overhead and pretty deterministic. Still, performing heap operations from real-time task should be done with care. While it's certainly not forbidden, it should be limited to a minimum, as it takes some time to be done.


Multicore handling

On multicore systems we differ between symmetric and asymmetric running systems. A system is called symmetric if the one binary is running on multiple cores. It is called asymmetric if different binaries are running the different cores. Currently only COP-MAS2 based systems are able to run asymmetric.

Symmetric running systems are :

  • running INOS with INOS_MULTICORE support on a GIN-SAM4 Quad
  • running INOS with INOS_MULTICORE support on a GIN-PCIe
  • running INOS with INOS_MULTICORE support on a IMP-MAS4
  • running INOS with INOS_MULTICORE support on a COP-MAS2 if both cores are used by the same system

Asymmetric running systems are :

  • application running on core 0, motor controller running on core 1 on a GIN-AX4x4 Pro
  • application running on core 0, motor controller running on core 1 on a GIN-SAC4xX Pro
  • application running on core 0, motor controller running on core 1 on a COP-MAS2

INOS implements symmetric multicore functionality by creating an own kernel instance for each core at startup. Tasks are always running on a fixed core. INOS does not try to optimize CPU power by moving tasks from one core to an other. It is in the responsibility of the user to define which task is running on which core, see also Multicore.

The only exception are GinLink based systems, where one can configure the GinLink to be distributed to all available cores. This can be done by setting Bit 6 in OptionsEx in ginlink.dt2.

OptionsEx ; uint16 ; 0x0040; // 0x0040-use all cores

Debugging multicore systems is currently still some kind of rudimentary. Each core is handled as an own target. So if you want to debug a task on a different core than 0, you have to select the corresponding core target first in iDev, e.g. MyTarget@1 for core 1, MyTarget@2 for core 2, ...