Adama Tutorial¶
Warning
This API is preliminary, and it may change.
Follow this tutorial with any HTTP client. Here we’ll use curl.
The languages supported by Adama are currently: Python, Javascript.
The base url of the Adama is https://api.araport.org/community/v0.3. For brevity, in what
follows we use the environment variable API
to refer to this base.
Declare the variable in your shell to be able to copy and paste the
examples:
$ export API=https://api.araport.org/community/v0.3
You need to obtain a token through your method of choice. In what
follows, the environment variable TOKEN
is assumed to be set to
such token. It is convenient to set it as:
$ export TOKEN=my-token
Checking access to Adama¶
A GET
request to $API/status
should return:
$ curl -L -X GET $API/status -H "Authorization: Bearer $TOKEN"
{
"api": "Adama v0.3",
"hash": "6869fde8e2617ab8f8a58c5c09b1512a80185500",
"status": "success"
}
The hash
field points to the git commit of Adama that is currently
serving the response.
Registering a Namespace¶
Namespaces allow Adama to group adapters. Create a new namespace with:
$ curl -X POST $API/namespaces -Fname=tacc -Fdescription="TACC namespace" \
-H "Authorization: Bearer $TOKEN"
{
"result": "https://api.araport.org/community/v0.3/tacc",
"status": "success"
}
Retrieve information about a known namespace from the url
$API/<namespace>
(for example $API/tacc
). Obtain the list of
all registered namespaces with:
$ curl -X GET $API/namespaces -H "Authorization: Bearer $TOKEN"
{
"result": [
{
"description": "TACC namespace",
"name": "tacc",
"url": null
}
],
"status": "success"
}
Delete a namespace with the verb DELETE
to the url
$API/<namespace>
.
Registering an Adapter¶
Adama currently supports two types of adapters: query
and
map_filter
.
A query
adapter receives a request through Adama, performs a query
to an external service and returns the results as JSON objects.
A map_filter
adapter transforms and/or filters JSON objects
returned from an external service.
An adapter can be registered using two methods (or a combination of them):
POST
the code and the metadata. The code can be a single file, tarball, or zip archive.POST
an URL to a git repository containing the code and the metadata.
It is strongly recommended to use the second method, since it makes it easier to share, to modify, and to keep track of changes in the adapters.
We show an example of a query
adapter registered via the first
method, and an example of a map_filter
adapter registered via the
second method.
Writing a query adapter¶
Write a Python module main.py
, with a function search
that
takes a JSON object as argument in the form of a dictionary. Print
JSON objects to standard output, separated by the characters
"---"
.
For example:
# file: main.py
import json
def search(args):
print json.dumps({'obj': 1, 'args': args})
print "---"
print json.dumps({'obj': 2, 'args': args})
This function can be tested in the Python interpreter:
>>> import main
>>> main.search({'x': 5})
{"args": {"x": 5}, "obj": 1}
---
{"args": {"x": 5}, "obj": 2}
Registering¶
To register this adapter with the name example
in the namespace
tacc
, we POST
to $API/tacc/services
with the metadata and
the code. In this example we show only some of the optional fields,
refer to the API docs for the full documentation.
- name (mandatory): the name of the adapter (
example
in this case), - type (mandatory): the type of adapter:
query
, ormap_filter
, - version (optional): version (default
0.1
), - url (mandatory): URL of the external service
(
http://example.com
in this case), - notify (optional): URL to notify with a POST request when the adapter is ready to use,
- code (mandatory): module
main.py
.
Using curl:
$ curl -L -X POST $API/tacc/services \
-F "name=example" -F "type=query" -F "url=http://example.com" \
-F code=@main.py -F "notify=https://my.url" \
-H "Authorization: Bearer $TOKEN"
{
"message": "registration started",
"result": {
"notification": "https://my.url",
"search": "https://api.araport.org/community/v0.3/search",
"list": "https://api.araport.org/community/v0.3/list",
"state": "https://api.araport.org/community/v0.3/example_v0.1"
},
"status": "success"
}
At this point the registration procedure is started in the server. It may take some time, and in the meantime the state of the adapter can be checked with:
$ curl -L -X GET $API/tacc/example_v0.1 \
-H "Authorization: Bearer $TOKEN"
{
"result": {
"msg": "Workers started",
"service": null,
"slot": "busy",
"stage": 4,
"total_stages": 5
},
"status": "success"
}
When ready, Adama will post to the url specified in the notify
parameter (if any), and the adapter can be seen in the directory of
services. To see a list of all the available services:
$ curl -L -X GET $API/tacc/services \
-H "Authorization: Bearer $TOKEN"
{
"result": [
{
"code_dir": "/tmp/tmpolAjqz/user_code",
"description": "",
"json_path": "",
"language": "python",
"main_module": "main",
"metadata": "",
"name": "example",
"namespace": "tacc",
"notify": "https://my.url",
"requirements": [],
"type": "query",
"url": "http://example.com",
"version": "0.1",
"whitelist": [
"localhost",
"example.com"
],
"workers": [
"57a4e10cb84aba5473d81c58011fcb78ce1b2684d67f0c2cc7540be191d4b589"
]
}
],
"status": "success"
}
Delete the service example_v0.1
by using the DELETE
verb to
$API/tacc/example_v0.1
.
Writing a map_filter adapter¶
Start a git repository as:
$ mkdir map_filter_example
$ cd map_filter_example
$ git init
Add the file main.py
with content:
def map_filter(obj):
obj['processed_by'] = 'Adama'
return obj
This module can be tested in the Python interpreter:
>>> import main
>>> main.map_filter({'key': 1})
{'key': 1, 'processed_by': 'Adama'}
Add also the file metadata.yml
with the metadata information:
---
name: map_example
version: 0.1
type: map_filter
main_module: main.py
url: https://api.araport.org/community/v0.3/json
whitelist: ['127.0.0.1']
description: ''
requirements: []
notify: ''
json_path: result
The url https://api.araport.org/community/v0.3/json
returns a sample JSON response:
$ curl https://api.araport.org/community/v0.3/json
{
"result": [
{
"key": 1
},
{
"key": 2
},
{
"key": 3
}
],
"status": "success"
}
The array of objects we want to process is in the field result
, so
we declare it in the json_path
field of the metadata file.
Commit both files into the git repository:
$ git add main.py metadata.yml
$ git commit -m "Add main and metadata"
The git repository has to be made available somewhere. For example, if
using Github with the username waltermoreira
and repository name
map_adapter
, we can register the adapter with:
$ curl -L -X POST $API/tacc/services \
-F "git_repository=https://github.com/waltermoreira/map_adapter.git"
Performing a query¶
Use the adapter example_v0.1
registered in the tacc
namespace
by doing a GET
from $API/tacc/example_v0.1/search
.
For example:
$ curl -L "$API/tacc/example_v0.1/search?word1=hello&word2=world" \
-H "Authorization: Bearer $TOKEN"
{"result": [
{"args": {"worker": "887e5cf7c82f", "word1": "hello", "word2": "world"}, "obj": 1}
, {"args": {"worker": "887e5cf7c82f", "word1": "hello"], "word2": "world"}, "obj": 2}
],
"metadata": {"time_in_main": 0.0001881122589111328},
"status": "success"}
Notice that the result consists of the two objects generated by
main.py
, including the query argument (in this
case containing some extra metadata added by Adama).
Use the adapter map_example_v0.1
in a similar way:
$ curl -L $API/map_example_v5/search \
-H "Authorization: Bearer $TOKEN"
{"result": [
{"processed_by": "Adama", "key": 1}
, {"processed_by": "Adama", "key": 2}
, {"processed_by": "Adama", "key": 3}
],
"metadata": {},
"status": "success"}
Summary¶
Current endpoints for Adama:
$API/status
GET
: get information about Adama server
$API/namespaces
GET
: list namespacesPOST
: create namespace
$API/<namespace>
GET
: get information about a namespaceDELETE
: remove a namespace
$API/<namespace>/services
GET
: list all servicesPOST
: create a service
$API/<namespace>/<service>
GET
: get information about a serviceDELETE
: remove a service
$API/<namespace>/<service>/search
GET
: perform a query
$API/<namespace>/<service>/list
GET
: perform a listing