The system consists of the following elements:
– Central server
– Docker registry compatible image registry managed by Megamind
– Computing nodes
– Developers
– End users

• Central server. Megamind Central server consists of the following parts:
  • Server REST API. Provides a top-level API for task management and connecting computing nodes. The task in this context is a docker image deployed to container(s) and run on a computing node. REST API server consists of an industrial-grade web server (apache / nginx) with https configured and accounts proxying calls to Megamind REST API server. The server supports the following commands:
  • Public part
    1. get a list of free computing nodes (id, hardware characteristics, computation cost)
    2. create a task by running an image from the registry on a particular node
    3. stop the task by id, destroying it and removing it from the list. It removes port forwarding from NAT and from the list.
    4. get a list of running tasks for the account (with id) and their state (started/pending/completed)
    5. pause / resume a task
  • Non-public part
    1. get parameters to connect the node to the system (parameters for entering swarm – token and ip)
    2. node registration: transfer to the server of its characteristics, ip, user:password
• The server also contains a manager node, a null / manager node docker swarm. The REST API server executes requests by managing the internal infrastructure using docker swarm (currently using parser of command line output, in the future versions – through the Docker API):
  1. getting a list of worker nodes: sudo docker node ls

For for more information on node:

2. task creation: sudo docker service create
3. deleting the task: sudo docker service rm
4. get a list of working tasks for the account: for each node getting all tasks via docker node ps and filtering by id for of this account
5. pause / resume task

With port-forwarding:

• it performs port forwarding to each created container (command for iptables);
• keeps track of already forwarded ports;
• removes port forwarding if necessary;

Or (option with direct channels to each node):

If your nodes are on a white IP and accessible from the Internet, then you need to open a port in the firewall (and don’t forget port forwarding in docker)

• Server of the web interface for setting tasks;
  
  

• Registry of images managed by Megamind GMbH

Fully compatible with standard docker registry. Needed authentication with htpasswd – every account registration on server from item 1 should add an entry to htpasswd. Allows to:

• unload (push) images
• download (pull) images
• get a list of images associated with this account
• delete images

All this using the standard docker registry interfaces.

• Nodes with computing equipment. High performance computers running MindOS.

Docker is installed on the node and the node is added into the swarm.

• Developers

They work under operating systems that support docker images, creating images for the Image Registry and load them into the Registry. They don’t work with Megamind-specific code or

Software — all work is done using docker tools.

• End users

They perform task management using direct API requests from clause 1.1 or using web interface: start new tasks, view running ones, stop, etc. They can also administer the counterpart of the computing Software (what requests / receives results from the nodes). Interface for accessing the task is ssh. If necessary, port forwarding is also done using ssh.