25. Advanced

25.1. Command line startup process

Here is the description of what happens after you execute veles:

  1. A new instance of __main__.Main class is created and it’s run() method is called. If run() returns, the resulting value is used as the return code for sys.exit().
  2. “Special” command line arguments are checked for presence. They include --help, --html-help and --frontend.
  3. argparse-based parser is initialized and parses sys.argv. See Command line arguments organization.
  4. Logging level is set (see -v / --verbosity option). DEBUG log level is set for classes enumerated in --debug.
  5. Random generator is seeded with the value of -r / --random-seed option, if it was specified. Otherwise, 16 numbers from /dev/urandom are used.
  6. Pickle debugging is activated, if --debug-pickle was specified (see Pickling in Veles).
  7. Peak memory usage printer is registered on program exit.
  8. The specified workflow file is imported.
  9. The specified configuration file is imported. Command line overrides the current configuration tree (see veles.config.Config). If --dump-config option was specified, the resulting configuration is printed to standard output.
  10. Special configuration parameters are parsed from command line and configuration file is override.
  11. If --dry-run was set to “load”, execution finishes just after instantiation.
  12. Otherwise, if -b / --background was passed, the process turns into a daemon process, which is independent of it’s parent. Many operations are performed in that case, most notably fork()-ing.
  13. If --optimize was passed, the parameters optimization procedure begins via the genetic algorithm, see Automatic optimization of workflow parameters.
  14. If –ensemble-train was passed, training of ensemble models is started.
  15. If –ensemble-test was passed, evaluation of ensemble models is started.
  16. Otherwise, run() is called from the specified workflow module. Two Main methods are executed _load and _main
  17. _load() constructs an instance of Launcher creates the workflow (corresponding class is specified as the first function argument) or restores it from the snapshot file (--snapshot, see Snapshotting). In the former case the launcher is passed as workflow’s parent workflow, in the latter case the “workflow” property is set to the launcher. If --workflow-graph was passed, writes the workflow scheme using Graphviz.
  18. _main() returns at once if --dry-run was set to “init”, otherwise it creates an OpenCL/CUDA device handle, unless --force-numpy was passed. Then it calls initialize() workflow’s method, passing in the created OpenCL device. If --dry-run was set to “exec”, returns. Finally, it calls launcher's run().

25.2. Veles Launcher

Basically, Launcher mimics some Workflow properties and methods, notably run(), stop() and add_ref(). This class is responsible for running Server (in master mode), Client (in slave mode) and the workflow. Besides, it starts GraphicsServer, unless -p '' (empty Matplotlib backend) was specified in the command line and GraphicsClient, unless --no-graphics-client was specified. See Graphics and plotting for more information about how plotting subsystem works.

Your workflow instance’s workflow property always points to Launcher object, even in __init__() (after super() call).

Launcher defines three modes of execution: standalone (default), master and slave. Each mode can be tested with corresponding properties: is_standalone(), is_master() and is_slave(), which are mutually exclusive.


25.3. Command line arguments organization

Every class which has CommandLineArgumentsRegistry as it’s metaclass (UnitCommandLineArgumentsRegistry for classes derived from Unit) can define it’s own command line arguments which are included into the global list. Such class must have


defined, which first obtain the instance of argparse.ArgumentParser

parser = kwargs.get("parser", argparse.ArgumentParser())

and then append arguments with parser.add_argument() and return it. When an instance of that class wants to find out the argument values, it calls init_parser() without any parameters and then invoke parse_known_args().

Internally, CommandLineArgumentsRegistry metaclass keeps the list of classes with additional command line arguments and during the startup each class from that list sequentially alters the same parser.


25.4. Pickling in Veles

pickle2 module was specially designed to easen proper pickling and unpickling throught Veles source code.

It defines best_protocol which should be passed into pickle.dump() as protocol value. The reason of this constant is to make pickle.dump() use protocol version 4 on Python 3.4 (it uses 3 by default).

setup_pickle_debug() patches pickle.dump(), pickle.dumps(), pickle.load() and pickle.loads() to use Python implementation instead of the native one, so that pickling/unpickling errors are printed with nice stack traces, the guilty object is revealed and pdb session is triggered.


Pickles debugging can be activated via --debug-pickle command line option.


25.5. Using memory.Array

To simplify interacting with OpenCL devices, special class veles.memory.Array was designed.

In order to use it:

  1. Import the Array class
  from veles.memory import Array
  from veles.numpy_ext import roundup

:func:`roundup() <veles.numpy_ext>` may be helpful for specifying ``ndrange`` when executing OpenCL kernels.
  1. In constructor or in initialize() create required vectors
  self.a = Array(numpy.zeros([512, 1024], dtype=numpy.float32))
  self.b = Array()
  self.b.mem = numpy.zeros([1024, 1024], dtype=numpy.float32)

Constructor of :class:`Array <veles.memory.Array>` may receive a numpy array, or numpy array can be assigned
directly to the :attr:`mem <veles.memory.Array>` of the :class:`Array <veles.memory.Array>` instance.
  1. Before supplying the vector to an OpenCL kernel (at the end of Unit’s initialize() when implementing the custom Units) call initialize()
  self.b.initialize(self, False)

:meth:`initialize() <veles.memory.Array>` receives instance of :class:`OpenCLUnit <veles.opencl_units.OpenCLUnit>` as the first argument
with optional argument ``bufpool`` (set it to False for vectors which have to live independently of the workflow,
for example ``weights`` and ``bias`` but not ``input`` or ``output``).
OpenCL buffer may or may not be created in :meth:`initialize() <veles.memory.Array>` depending on ``bufpool`` value.
  1. Assign vectors to OpenCL kernels if necessary, this should be done usually in ocl_init()
  self.krn_.set_arg(0, self.a.devmem)
  self.krn_.set_arg(1, self.b.devmem)

:attr:`devmem <veles.memory.Array>` is the OpenCL buffer handle.
  1. Just before executing the OpenCL kernel call unmap() on the vectors it uses
  self.execute_kernel(global_size, local_size, self.krn_)

:meth:`unmap() <veles.memory.Array>` transfers data to OpenCL device from CPU address space only if it was mapped before,
so it safe and fast to call it multiple times.
  1. Before you want to use vector’s data on the CPU, you have to call: map_read() and then use the data in read-only manner, map_write() and then update the data, map_invalidate() and then completely rewrite the data without caring for what it was in it before.