Are you ready for the Clicker generation ?

Dlods and API

What is a DLOD ?

The concept of DLODs is a cruxial point in the Clicker system. The main goal of dlods is to allow an application program to import some objects from servers programs or library.
These objects will then be used either as an interface to the server program or to provide a complete service. (i.e. plugging in some module into the app) The main particularity of dlods is that it really embeds some external code run-time into an application. (Note that, as Dlods are objects, you also can use that mechanism to import datas)


Where do DLODs come from ?

So that's nice: let's say it's possible to include some external code inside an application and use it as if it was local code... But where the hell will my prog. find those dlods!? There are two possibilities.

First of all, (and it's from far the easiest way), the dlod can be found in a library on a file. You can then ask to load the whole library, or a single object, or even a package of dlods (This assumes dlods are grouped in package inside of the library).

On the other side, the dlod mechanism allows a server program to make some code available to clients. So you can export functions that will be used by the client program to manage the interface object. As in the library approach, you can export packages of dlod (eventually, mixing some dlod you provide and some other dlods you just pass from other sources ;-)


Naming the Dlods and the other objects

Each dlod is suppose to have a 64 bits unique identifier within its package. The package itself has a similar 64 bits unique identifier.
To find out a specific dlod, you'll have to give a name like 'library:package/dlod'. The library name is a 16 characters-wide string. Each new library in the system has to be registered with this name. The system will then make a link between the libname and the filename. Registering will also need some extra informations (like the required trust level for this library).
Once the system has choosen the library file matching with the program parameters and the given libname, It will then look after the package name. The library provides an index of the available packages (by name) it holds. Note that this name needn't to match the package uid.
The package uid (the answer to the query for 'lib:package') can then be used for importing, exporting (if you has privilege enough) dlods or simply can be transmitted to sons or friends for their personnal use (if they have at least the same trust-level as the "giver").


Using Dlods in the API

Every program that is loaded by the Clicker will get a package of dlods that provide principal functions (like calling other packages, and communicating with other programs). The program will specify (by a table that is processed at load-time) which dlod it will need in that package and how it expects these dlods to be embedded into the program.
For example, you would get a "mailbox" dlod that would provide actions such "get_mail()", "mail_count()" and "send(mail)".
The advantage of working with packages of dlod is that package may be prepared to be loaded completely. So if the components inside of the package need each other, you won't have to link them each other before embedding them into the host.

The main idea behind the dlod technology is that every program is given a trust-level. This trust-level isn't static: each time the program is ran for an enough long time, it is given a trust point. When it has enough trust points, the program can increase its trust level. A program with a higher trust level will get more dirrect (and thus faster, but less protected) access to some ressource (like his mail, or services, or anything else...)
But if the program crash, it will return at trust level zero ("bugged" level). With the same idea, the system decreases the trust points counter of every other program running when the bad program crashed. If the system hangs (and if it is only due to software), every program that was running at that time go one trust level down.

The maximum trust level each class of program may reach and the number of points/level programs are decreased at crashes are fixed by the sysop.
As said above, the trust level can change the choice of the packages that are given to programs, so, by writing many packages that provides the sames services with different "trust level", it is possible to make programs running "as fast as it don't crash the system".
For example, at trust level 0, each dlod (and each object it makes) should be in a separated segment and the program has to use call gates to access them. Moreover, the dlod will check the program has the rights enough to invoke so calls to the system/server.
But at the "confiant" level, the same dlod is in the main code segment and it creates its objects in a shared-area of the program's heap. So methods can be used as a local call manipulating local datas.


Dependencies Checks (and other boring stuffs)

Up to now, we've supposed that the dlod we loaded were self-sufficient. But in general, dlods may need other dlod to work. And of course, it would be great if the loading of needed package was transparent to the host.
In fact, it wouldn't be fun if, while asking for a dlod, your application got error message like "package #12246 not present, please load it before".

So, a package will provide a list of all the other package it could need, and also a list of head-dlods. Only head-dlods may be asked separately from the other dlods inside of the package (and they have a list of ressource they need to be joined, i.e. the other same-packaged dlod or external package needed if you just want to load THAT chunk of dlods.)
When loading a chunk of dlods, the system will first determine what other package are required (this will take some time in looking libraries, etc.) If the package cannot be loaded (i.e. if one of the required package doesn't exist), the system will abort its attemp and try to load the same package from a twin-library.

If no twin-library succeed to load the package (and its friends), the call will fail with a message like "unable to find friend-package". Another approach is to abort loading at the first miss and giving the name of the missing package.

By the other way, at each successful load, the system builds a "temporary" package where it prepares each dlod with each other. So when every dependency has been processed, you got a "temporary" package in the system that is ready to use. This temporary package gets a brand-new package uid, which is linked in a "cache" with the name of the loaded chunk of dlods and the trust level needed (so that if other programs need the same chunk of dlods, the system won't need to do all the dirty job a second time :-)


Creating packages and exporting them

As i say upper, "servers" should be able to create new package. Therefore, they first create an "unnamed" empty package, and then they'll fill it with dlods.
That package is said "unnamed" because it's impossible to reference it with a classical name such 'library:package', but it has still a package uid. This uid is taken in the "temporary" idspace, so we are sure it won't interfere with another package in a library. The programs that creates such a package can keep it to give the same package (with the same uid) to many clients.

The second step is to fill the package with dlods. Dlods are made of pieces of code and data. You can then build dlods from scratch by addind regions one by one. Each region having a start address in the server's space, a lenght, a class (data/code) and a relocation list (i.e. a dependencies list).
When adding a dlod into a package, the regions are copied into the package's space and are relocated into that space (i.e. the code is realigned). Moreover, the relocation list is processed (to find out if we can find some dependencies of the added dlod into allready present dlods or vice-versa) and the new depen -dencies are added to the package dependencies list.

It's clear that such a scheme is quite time-consuming, so it should be possible to make ready-to-use package image from an object file of a compiler rather that building it at run-time.
When merging two packages, we will first group all regions and the merging and simplifying dependecies list (this should be more efficient than adding them one by one). It's widely recommended to merge pre-build package rather than building them from scratch everytime it's possible to do so.


Some Post-Scriptum consideration...

  • When asking for a package (even if this package is required to import another one), the system should (and must) first check if that package isn't already loaded for the host program. This means each program must update a table where it collects infos about all loaded packages (i.e. which packuid has been loaded, which head-dlods are available and where it has been put in memory,etc)

  • The Dlod technology assumes there will be few objects imports/exports and many more use of imported objects. So it makes import a little bit slow (than the speed it could have had if the imported object had been put in separated segments), but definitely boosts-up the imported objects use (because they have been embedded by the system in the "local" code and data segments, so you need quite few overhead to access them.
  • The Dlods may be convenient for exchanging classes rather than instance of objects. This means once the import has been done, you'll have to call a builder of the class to make objects. If these objects are interfaces with a server, it's a good idea to include code to register them by the server in the constructor call.
  • The way the program uses the dlods may affect load-time efficiency. For example, it should boost up the linking to refer a table where every dlod's address is stored rather than using dirrectly the dlod's imported address into the program's code. This is because the first scheme makes less references to the imported object (only one, in the table) and thus you'll have less relocations to performs when linking modules together. (This scheme will be used for the api default dlod package).

  • Pfioouuh...
    I think it will be all for this time. If you're interrested in the dlod technologies, i suggest you post me an e-mail with your doubts, improvements or thinkings about it.
    Please keep in mind that the dlods are a product of tons of hours of intensive reflexions for me, so if you want to use such a mechanism in another os, i'd be please to be informed from it and it would be really nice to credit me.

    I hope this doesn't sound too much like OLE and friends.