Skip to content

Developing plugins for Expedient

Jump to bottom
Carolina Fernández edited this page Apr 1, 2014 · 22 revisions

Home > Developing plugins for Expedient

This manual is intended for AM developers that want to integrate the data returned by the Aggregate with the Expedient GUI.

The idea

The plugin system is a way to ease development of new Aggregate Managers and its interaction with the GUI (Expedient).

  • Aggregate Manager (AM): manages resources of a given kind
  • Plugin: communicates with the AM to retrieve resources according to some logic

Therefore, a developer may write a plugin to see the AM resources within the Expedient topology (the slice detail page).

The communication between Expedient, the plugins and their corresponding AM is as follows:


    ------------------------------------------------------ <- [3] translates node and link structures  
    |                      Expedient                     |        to show topology
    ------------------------------------------------------          
          |                |                   | <----------- [1] calls to each AM plugin method 
          v                v                   v                  (get_ui_data) to retrieve data
    ---------------  ---------------    ------------------
    | plugin <vt> |  | plugin <of> |    | plugin <yours> |
    ---------------  ---------------    ------------------
          ^                ^                   ^
          |                |                   | <----------- [2] each AM returns nodes and links   
     ------------     ------------        -------------           through its internal implementation
     |   VT AM  |     |   OF AM  |        |  your AM  |
     ------------     ------------        -------------

The plugin system was first released along with OCF 0.5.

How to develop a plugin

Folder location

Folder must be placed under /opt/ofelia/expedient/src/python/plugins/

The name of this folder will be used to designate the plugin.

Folder structure

Plugins follow a minimal folder structure that allows the PluginLoader to retrieve the data

    |-- <plugin_name>           Root folder
    |   |-- settings.conf       Configuration file
    |   `-- urls.py             URLs file (as in any Django package)

What to fill

File: settings.conf

The settings file contains a number of settings to be read from Expedient that you will have to configure to your own needs. You may also add new settings and use these within your plugin.

An explanation of each setting follows:

  [urls]
    BASIC_AUTH_URLS:       URLs that need basic authorization to be accessed

  [paths]
    CSS_DIRS:              relative path to folder where CSS files are stored within your plugin
    IMG_DIRS:              relative path to folder where images are stored within your plugin
    JS_DIRS:               relative path to folder where JS files are stored within your plugin
    TEMPLATE_DIRS:         relative path to folder where HTML files are stored within your plugin
    TEMPLATE_RESOURCES:    relative path to the file that allows to add resources to the slice.
                           This will be shown as a collapsible container in the slice details page

  [general]
    AGGREGATE_PLUGINS:     (list of) 3-tuple(s) where:
                             1st: model for the resource aggregate. This defines the fields in the
                                  "Add aggregate" form in the initial view of Expedient
                             2nd: name of the plugin
                             3rd: location of the URLs file (not a relative path; use
                                  <plugin_name>.urls for example)
    INSTALLED_APPS:        (list of) Django app(s) to be activated. You should set -at least- your app
                           name
    GET_UI_DATA_LOCATION:  defines the file where the method "get_ui_data(slice)" is defined
    RESOURCE_TYPE:         kind of resource, currently identified by {"network", "computation"}. This
                           is useful to distinguish diferent resources and group these accordingly in
                           the slice details page (e.g. virtualization servers and sensors will be
                           under the "computation" category, while switches will fall under "network")

Method: get_ui_data(slice)

Each plugin has to implement the method "get_ui_data(slice)" and "link" the file where it is contained through the settings.conf file.

This method processes the data and return a dictionary with (at least) the keys "nodes" and "links". The values for those keys are a list of Node's and a list of Link's, respectively.

def get_ui_data(slice):
    return {"nodes": [Node(...), ...], "links": [Link(...), ...]}

Node and Link are extendable structures that contain a basic skeleton that must be filled:

Node
    name: node name
    value: node (unique) identifier
    description: html-formatted data to be shown in a tooltip when hovering the node in the topology
    type: type of resource (e.g. "OpenFlow switch", "Sensor device"). Any text you like
    image: path to the icon used to show the resource node in the topology. You can get the path in
           this way: reverse('img_media_<plugin_name>', args=("<file_name>",))
            This path is the one defined in the IMG_DIRS setting for your plugin

Link
    source: source node ID
    target: target node ID
    value: string with a specific format depending on:
           * "simple" nodes connected directly to other (e.g. switches, sensors):
             "rsc_id_<x>-rsc_id_<y>" where:
                       - {x,y} = node IDs used in Django
           * "complex" nodes (e.g. servers) connected through a given interface to a "simple" node:
             "rsc_id_<x>-<y>:<z>" where:
                       - x = "simple" target port ID assigned to Django resources (e.g. 355)
                       - y = interface name (e.g. "eth1")
                       - z = "simple" target port number (e.g. 65534)

You may pass more arguments to each node and link if you want some extra data to be used in your plugin templates:

Node(
     name = "Bodhisattva",
     value = "251",
     description = "<strong>Sensor: Bodhisattva</strong>
<strong>Connections</strong><ul><li>Sensor 249</li><li>Sensor 252</li><li>Sensor 253</li></ul>",
     type = "Sensor resource",
     image = reverse('img_media_sample_resource', args=("sensor-icon.png",)), 
     connections = [249, 252, 253]
    )

Link(
     source = "251",
     target = "252",
     value = "rsc_id_251-rsc_id_252"
    )

File: urls.py

Fill this as in every URLs file for a Django package: urls to create/edit/delete both aggregate and resources, etc.

Everything else

You can add any other file and logic to your plugin. Just keep in mind to fill the gaps as stated to enable the communication. Everything else is up to you :)

Known limitations

  • The templates for every plugin are tried in order from the paths defined in the Django setting TEMPLATE_DIRS, so if two templates with the same name exist in different plugins only the one in the first path will be loaded. To avoid this and to establish a common style, prepend your plugin name to every template (e.g. plugin "sample_resource" with template "add_resources.html" would be renamed to "sample_resource_add_resources.html")
  • The monitoring of each aggregate manager' status is done by retrieving its address from the aggregate model (defined at the time of its creation). This requires at least the url field from the xmlrpcServerProxy model to be present

TODO list

  • Improve plugin setting loading (plugin settings should be accessible at any time)
  • Integrate with [Theme Manager](Theme manager) to allow different templates
  • Establish a generic format for links between nodes - without differentiating between "simple" (OF switches) and "complex" (servers) nodes (see [Link explanation](Developing plugins for Expedient#method-get_ui_dataslice))

Testing with the sample plugin

How to enable it

  1. Install dependencies

    apt-get install python-lxml

  2. Move the code from /opt/ofelia/expedient/doc/plugins/samples/ as following:

  3. Move plugin (plugin/sample_resource) under /opt/ofelia/expedient/src/python/plugins/

  4. Move AM (aggregate/sr_manager) under /opt/ofelia/

  5. Synchronize database:

    cd /opt/ofelia/expedient/src/python/expedient/clearinghouse
python manage.py syncdb

    Output should be similar to:

    Creating tables ...
Creating table sample_resource_sampleresource_connections
Creating table sample_resource_sampleresource
Creating table sample_resource_sampleresourceaggregate
Creating table sample_resource_xmlrpcserverproxy
Installing custom SQL ...
Installing indexes ...
No fixtures found.

  6. Restart Apache

  7. Configure the SR AM if you need it (defaults are fine, though) in the file /opt/ofelia/sr_manager/src/settings.py.

  8. Go to the SR AM folder and start the server:

    cd /opt/ofelia/sr_manager/src/
python server.py

    Note: if by any chance you do use Python2.7 and find that your server crashed along with an error like TypeError: shutdown() takes exactly 0 arguments (1 given), the werkzeug library needs to be fixed (see AMsoil wiki).

    Go into /usr/lib/python2.7/SocketServer.py and add the following below line 461:

      except TypeError: # << add this
      pass # << add this

    Resulting method will be as follows:

    def shutdown_request(self, request):
  """Called to shutdown and close an individual request."""
  try:
      #explicitly shutdown.  socket.close() merely releases
      #the socket and waits for GC to perform the actual close.
      request.shutdown(socket.SHUT_WR)
  except socket.error:
      pass #some platforms may raise ENOTCONN here
  except TypeError: # << add this
      pass # << add this
  self.close_request(request)

  9. Create an aggregate manager with type SampleResource in Expedient. E.g.:

    Name:					SR AM
Description:			SampleResource Aggregate Manager
Geographic Location:	Barcelona
Sync resources?:		x
User:					user # Default value
Password:				password # Default value
Server URL:				https://<IP_OR_DOMAIN>:9445 # Default port

  10. Add this AM to a project and then to some slice within. You should find now five "sensors" conforming a pentagon-shaped topology

How to disable it

  1. Delete any AM of the SampleResource type (via the Expedient welcome page, in the AM list). This shall delete every AM and also any SampleResource associated to it

  2. Stop the SR AM server

  3. If you do not want to completely remove data for this plugin/application jump this step.

    Otherwise type the following:

    cd /opt/ofelia/expedient/src/python/expedient/clearinghouse
python manage.py sqlclear sample_resource

    and execute the code returned inside your MySQL engine to drop the corresponding tables.

    cd /opt/ofelia/expedient/src/python/expedient/clearinghouse
python manage.py dbshell
mysql> (paste previous SQL code here)

  4. Move the code back into their corresponding folders, under /opt/ofelia/expedient/doc/plugins/samples/

  5. Restart Apache

Note: if you do not follow the previous steps before moving the sample plugin code you may find that the AM list in the Expedient welcome page is not loading. If so, place the code back in the plugin folder and follow these steps.

Troubleshooting

Error ImportError: No module named urls

You might still have the old plugins (openflow, vt_plugin) under /opt/ofelia/expedient/src/python. Backup these into another directory or delete them, as you wish. Then reload Apache and try again.

Errors: 'str' object has no attribute '_meta', KeyError: 'request'

There is an Aggregate Manager that couldn't be removed from the MySQL database. Enter your MySQL engine and type the following (or look for it yourself) in order to delete the stale Aggregate Manager:

use expedient;
select id from aggregate_aggregate where leaf_name = 'SampleResourceAggregate';
delete from aggregate_aggregate where id = <previous_id>; # Type here the ID obtained previously

Error Unknown column 'username' in 'field list'

This may happen because you installed the version from the ofelia.stable branch and your Expedient database contains some stalled tables that were later modified. You should delete your old tables structure and synchronize again. For that:

  1. Follow step #3 only from the disable section
  2. Follow step #4 and onwards from the enable section
Clone this wiki locally