Avorion Internal Architecture

From Avorion Wiki
Jump to: navigation, search

Entity-Component System

Avorion uses an Entity-Component System for its internal architecture. There are objects in the Scripting API for Entity and several components.

Developer Comment: "Note that we are currently working on exposing all components to scripts, but haven't managed to do all yet."

In order to enable multithreading, during the update step, each sector is hermetically sealed from all other sectors and can only be accessed through special functions. This is only relevant on the server though, the client always only knows about the sector the player is in and cannot access other sectors anyways. Check out the section below on server architecture to learn more about sectors and inter-sector communication.

Scriptable Objects

In Avorion, most scripts don't exist in a vacuum, but are attached to an object. At the time of this writing, those objects can be: An Entity (ie. asteroid, ship, station, etc.), a Player, an Alliance, an AIFaction or a Sector. Those are called scriptable objects.

You should attach your scripts to those objects that you want to influence most. If it doesn't really work out and you need to influence more than one kind of scriptable object, consider writing multiple scripts.

To attach scripts, you can call the addScript() or addScriptOnce() functions those objects provide:

-- add a the pirate attack event script to the current sector
local sector = Sector()
sector:addScriptOnce("data/scripts/events/pirateattack.lua")

You can add scripts on initialization to objects by extending the init.lua script of those objects. init.lua of a scriptable object is executed whenever a new scriptable object of that type is created.

Predefined Functions

Scripts that are attached to Scriptable Objects can define a range of predefined functions, that will be called by the game at various points in time. You can find more information about all existing predefined functions in the Scripting API.

Examples

Some typical examples for scripts attached to various scriptable objects:

  • Entity: Ship behavior scripts (ie. AI, Xsotan attack behavior, Bosses), Station scripts (ie. Trading Post, Resource Trader, etc), Object Behavior (ie. Claimable Asteroids, Secret Stashes, etc)
  • Player/Alliance: Missions (ie. story missions, kill missions), Player Debuffs (Avorion doesn't have this at the time of this writing, but it might be a possibility)
  • AI Factions: AI Faction Behavior
  • Sector: Sector events, event scheduling (ie. pirate and xsotan attacks and their timing), Sector Rules (ie. Neutral Zone)

Special Behaviors

Your scripts are kept alive as long as the object is around that the script is attached to. When an entity is deleted or destroyed, the script is removed. When a sector is unloaded and saved to disk, its scripts are saved to disk as well, same for all entities that are inside the sector. When a player logs out, all of his scripts are saved to disk as well and no longer running.

For mods, scripts that are attached to Entities, Players, Alliances or Sectors must be present on both the client and the server, so tagging your mod as serverSideOnly is not just a bad idea in those cases, but will also cause errors during runtime.

These are some special behaviors that you should know about:

Server

  • Player scripts will travel with the player when he changes sectors
  • Player scripts are only active while the player is logged in
  • Entity scripts will travel with the entity when it changes sectors
  • Sector scripts never change sectors
  • AIFactions are server-only

Client

  • Player scripts will travel with the player when he changes sectors
  • Sector & Entity scripts are deleted on sector change (and then re-initialized) because the client-side sector is destroyed and then re-initialized with new data received from the server
    • The scripts for the player's ship are also deleted and re-initialized
  • AIFactions are server-only

Predefined Function Call Order

These are some of the call orders that you can rely on. Check the Scripting API for more detailed information on the order of those functions.

  • initialize() is always called as the very first function, even when restoring from disk, restore() will be called after initialize()
  • secure() will always have been called at some point in time before a restore()
  • getUpdateInterval() will always be called before any update() function
  • Order of the update functions: updateParallelRead(), updateParallelSelf(), update(), update[Server/Client]().

Access to Parent Objects

Every script that is attached to a scriptable object has access to that object as well as the contextual objects of that object. For example, a script attached to an Entity has access to that Entity and the Sector the entity lives in, as well as other entities in that sector. It doesn't, however, have access to other sectors than the one it lives in. Due to internal engine structure and asynchronous updating of sectors, a script can always only have access to a single sector, which is most often the sector that script is in. Scripts can however, have access to all factions, players, alliances and other server-related objects at all times.

To simplify scripting code, scripts have access to a few implied variables, that they can access without using an index. For example, scripts that are attached to an entity, can easily access their entity by calling Entity().

  • When a script is attached to a sector, it can call
    • Entity(id) to access other entities in that sector. It can also call
    • Sector() to gain access to that sector.
  • When a script is attached to an entity, it can call
    • Entity() to access itself,
    • Sector() to access the sector and
    • Entity(id) other entities in that sector.
  • When a script is attached to a player, it can call
    • Player() or
    • Faction() to get access to the player it is attached to. It can also call
    • Alliance() to gain access to that player's alliance, as well as
    • Sector() to access the sector that player is in.
  • When a script is attached to an alliance, it can call
    • Alliance() or
    • Faction() to get access to the alliance it is attached to.
  • When a script is attached to an AI Faction, it can call
    • Faction() to get access to the faction it is attached to.

VMs

A lua VM (Virtual Machine) is a lua state or lua environment into which functions, variables and scripts are loaded and created and executed. It's comparable to a miniature process inside Avorion. In order to interpret lua scripts, Avorion creates lua VMs that those scripts run in. Those are hermetically sealed lua environments, that can only access each other through special functions, similar to the sector structure on the server.

Avorion creates at least one lua VM per object with scripts attached. It does its best to only create a single VM that those scripts run in, in order to save memory and allow scripts to access each other, as long as they're attached to the same object. Since every script has to define the same functions, adding more than one script with the same function names would override functions previously defined by other scripts.

Namespaces

In order to avoid name-conflicts, Avorion uses namespaces, which are nothing more than a table that all functions and variables of the script should be defined in:

-- This comment tells Avorion that all functions in this file, that it can call from outside (see Predefined Functions in the Scripting API) live in the table "Shipyard"
-- namespace Shipyard
Shipyard = {}

function Shipyard.initialize() -- instead of just "function initialize()"
end 

-- with the above namespace defined, Avorion ignores this function. Use "function Shipyard.update()" instead.
function update() 
end

When the first script is attached to an object, a new VM is created and the script is loaded into it. All functions the script defines are then defined in the VM. Avorion looks through the file to find a comment like -- namespace Shipyard that defines a namespace.

When a second script is loaded, Avorion checks the namespace of the second script, and if there is no name conflict with previously loaded namespaces, it loads the script into the same VM as the previous one. When you add two scripts that define the same namespace, or even the same script twice, the second script will be guaranteed to be created inside a new VM.

When no namespace is defined, Avorion can't make assumptions about namespaces and always creates a new VM for that script.

Note: When a script is removed from a scriptable object, the namespace it defined is deleted (set to nil).

Accessing other Scripts/VMs

When accessing other scripts, you have to distinguish whether or not the script is loaded into the same VM or not. When accessing scripts from another scriptable object, the script will always be inside another VM.

Same VM

When two scripts are attached to the same scriptable object and have different namespaces and thus share the same VM, they can just access each other through their namespaces:

-- namespace SomeScript
SomeScript = {}
function SomeScript.doSomething(arg1, arg2)
    ...
end
-- namespace OtherScript
OtherScript = {}
function OtherScript.update(timeStep)
    if SomeScript and SomeScript.doSomething then -- check if the namespace and function exists in the VM
        SomeScript.doSomething(1, "argument 2") -- call the function in the other namespace directly, because they're loaded into the same lua environment
    end
end

Different VM

If two scripts are not attached to the same scriptable object, or have the same or no namespaces, they don't share the same VM. They can't access each other through their namespaces then and you have to call invokeFunction() on the other object. See the Scripting API for more information about invokeFunction().

In contrast to the invokeRemoteFunction() family, this function returns values from the called function, in addition to a value indicating whether or not the call was done successfully.

-- In this case, this script is attached to the sector (but it could be an Entity or Player as well; It could even be the same entity, if you don't define a namespace)
-- namespace SomeSectorScript
SomeSectorScript = {}
function SomeSectorScript.doSomething(arg1, arg2)
    ...
end
-- This script is attached to an entity in the sector
-- namespace OtherScript
OtherScript = {}
function OtherScript.update(timeStep)
    local sector = Sector()
    -- we must specify what script and what function we want to call. for more infos see the Scripting API docs
    -- calls the function in the other scriptable object, and returns a value if it worked, followed by all return values of the invoked function
    local ok, ret = sector:invokeFunction("somescript.lua", "doSomething", 1, "argument 2")

end

Note: This works for all scriptable objects (ie. Entity to Entity, Sector to Entity, Player to Sector, etc.

Other Kinds of Scripts

There are some other scripts, that are not attached to a scriptable object and may only be alive for a short time.

Chat Commands

Those scripts are only alive for a very short time, while the command is run. They have access use Player() to access the player that executed it, Entity() to gain access to the craft that player is currently flying and Sector() to access the sector that player is in.

System Upgrades

Those are attached to an entity and follow the same rules as any other entity script.

Sector Templates

Those scripts are only executed and alive while a sector is generated. They can also be included in various other files to gather information about all possible sectors.

They can call Sector() to get access to the sector that is being generated.

Note: These scripts have restricted access to some functions, since they're being run in a separate thread, completely independently from all other sector updates.

Items

Item scripts have access to Player(), Alliance() and Faction() to get access to the player (or alliance!) that is using them.

server.lua

This script doesn't have special implied access to any classes.

main.lua

This script doesn't have special implied access to any classes.

Client and Server

Avorion, being a multiplayer game, is separated into 2 parts: Client and Server. In order to keep network traffic to a minimum, clients run mostly independent from the server, only receiving updates every few frames. Clients predict the simulation until a server update comes in. So not everything is always synchronized between client and server.

As a general rule: All important gameplay-related changes should always happen on the server, to prevent cheating and to ease synchronization with connected clients.

When you add a script to a scriptable object, you can only do this on the server. The server will then synchronize the new script to all connected clients, so when you have 3 players in a sector, you will have 4 instances of that script running: 1 on the server and 3 on the client.

Client and server can have vastly different performance and update tick timing, so for most of the time the client instances of the script will run independently to the server instance.

Communication Between Client and Server Scripts

In order to communicate, you have use so-called remote procedure calls: The client then sends a network message to the server, telling it to execute a function, or vice versa.

There are a variety of functions that you can use to do a remote procedure call. This is only a basic listing, check out the Scripting API for more information.

-- on the client:
function invokeServerFunction(functionName, arg1, arg2, ...)
-- on the server: 
function invokeClientFunction(player, functionName, arg1, arg2, ...)
function broadcastInvokeClientFunction(functionName, arg1, arg2, ...)
  • invokeServerFunction() will call the given function on the server, in the server-instance of the script that is calling invokeServerFunction(). So when you're in, say, shipyard.lua on the client, calling this function, it will call the function on the server-instance of the entity, in the shipyard.lua script.
  • invokeClientFunction() does the same but vice versa for the server. But as there can be more than one player on a server, you have to tell it on which player's instance of the script you want to call the function.
  • broadcastInvokeClientFunction() does the same as invokeClientFunction, but invokes it for all players that are currently present in the sector.

Note: These function invocations don't include a return value of the remotely executed function. Also, you won't get a notification if the call succeeded or not. In order to get a response or return value, you'll have to do another remote invocation on the other side.

Note: See the Mod Example: Entity for an example on client server communication.

Server Architecture

The server has 2 kinds of big global databases where it saves its data: The faction database (players, alliances, ai factions) and the sector database.

The server updates its sectors using multithreading, with a frequency of 20 fps (ie. 50 ms per frame).

Sector Structure

The server has multiple sectors loaded at the same time. During the update step (see below) access to other sectors is not possible, resulting in scripts basically never being able to directly access another sector's content.

It is possible to access another sector indirectly, by using these functions (this is only a basic overview, check the Scripting API for more details):

-- invoke a function in a script that is attached to either the other sector or entity in the sector
-- these 2 functions are also available on the client and can be used to call a script on the server that's not in the same sector as the client
function invokeRemoteEntityFunction(...) 
function invokeRemoteSectorFunction(...)

-- send some code to be executed in a temporary pseudo-script that is either attached to the entity or sector
function runRemoteEntityCode(...)
function runRemoteSectorCode(...)

Those functions behave the same way that the invokeServerFunction() etc. functions work: They invoke a function asynchronically, don't return its value and won't return an error in case they failed. In order to get a response, you have to do another call of one of these functions back on the remote script that you're calling.

Note: Depending on the server configuration, it can take up to several seconds until these functions are actually executed in the remote sector.

Update Step

A single update step (frame) of the server can be roughly divided into a few parts:

Sequential Part

At first, the server updates all players and factions sequentially and does other administrative stuff like network updates, handling RCON commands, profiling and so on.

Parallel Part

This is where the real heavy lifting happens.

After the sequential part, the server does a sector update, where it updates all sectors in parallel, using as many threads as it got configured in the server.ini settings file. While doing the sector update, it doesn't just update all sectors in parallel, but also all entities inside those sectors are updated in parallel, to get the most speedup possible and to have as little waiting time as possible.

Since scripts are potentially unpredictable, their updates still have to be executed sequentially one after another in order to avoid race conditions. There are some exceptions though, where scripts can have special functions defined that restrict their access to other entities, but that can be executed in parallel. You can read up on predefined parallel functions here.

Note: During this update step, access between sectors is restricted, which is why all scripts can only access their own sector, and sometimes not even that (see the parallel entity script updates, for example).

Note: Important: Since it is not possible by its nature to have multiple threads inside a single lua VM at the same time, you should make sure that your update() and updateServer() and similar functions have good performance. Otherwise they will block the execution and thus stall the update rate of the server, because the server has to wait for the script update to finish before it can move on to the next part.

Wrap-Up and Loop

Once all updates of all entities of all sectors are finished, the server does a wrap-up and sends out required update messages. After that, the loop starts anew.

See Also