Datazilla is a system for managing and visualizing test data. It’s designed to be able to manage different subsets of test data, such as performance data. This project includes database template schemas, a model layer, web service, and web based user interface.
At a top level, performance test data is submitted as a JSON data structure to the webservice via HTTP POST, an example data structure can be found here. It’s stored in a dedicated database schema as an object. The objects in this schema are then translated into a relational database schema where individual fields in the JSON structure are indexed for queries. The indexed data is used for applying statistical analysis to identify performance regressions. Both the indexed data and un-processed objects are available through the web service.
New JSON objects and schema definitions can be added to support different types of data. It’s also possible to utilize the fields already available in the JSON structure to represent different information.
All of the datasets contained in datazilla are described with three classifiers: project, content type, and dataset. Each project classifier can have its own dedicated user interface associated with it. These classifiers are described in detail in the Architecture section.
Datazilla was developed to manage performance test data generated by talos. The user interface displaying this data is deployed here.
The web services available in datazilla can be grouped into the following categories.
These are a set of webservice endpoints for getting the raw test data for a given project.
Return a list of the test data for the project, branch and revision specified.
If a JSON blob of test data is malformed, or has an error, then a placeholder with the error message is returned.
| Query Parameters: | |
|---|---|
|
|
Example request:
GET /talos/testdata/raw/Mozilla-Beta/ebfad1bf8749/
GET /talos/testdata/raw/Mozilla-Beta/ebfad1bf8749?product=Firefox
GET /talos/testdata/raw/Mozilla-Beta/ebfad1bf8749?os_name=mac&test_name=Talos%20tp5row
GET /talos/testdata/raw/Mozilla-Beta/ebfad1bf8749?os_name=mac,linux
GET /talos/testdata/raw/Mozilla-Beta/ebfad1bf8749?page_name=digg.com,alipay.com,tmall.com
Example response:
Content-Type: application/json
[
{
"results_aux": { },
"test_machine": {
"platform": "x86_64",
"osversion": "OS X 10.6.8",
"os": "mac",
"name": "talos-r4-snow-074"
},
"testrun": {
"date": "1342729660",
"suite": "Talos tscroll.2",
"options": {
"responsiveness": false,
"tpmozafterpaint": false,
...
}
},
"results": {
"tiled.html": [
14522,
10408,
10414,
10414,
10372
],
"iframe.svg": [
11941,
11474,
12153,
12451,
11686
],
...
},
"test_build": {
"id": "20120719120951",
"version": "15.0",
"name": "Firefox",
"branch": "Mozilla-Beta",
"revision": "ebfad1bf8749"
}
},
{ … },
{ … },
{ … },
{ … }
]
These are a set of web service endpoints for retrieving metrics data.
Return all metrics data for the project, branch and revision specified. If no product name is supplied the product name defaults to Firefox.
| Query Parameters: | |
|---|---|
|
|
Example request:
GET /talos/testdata/metrics/Mozilla-Beta/ebfad1bf8749/
GET /talos/testdata/metrics/Mozilla-Beta/ebfad1bf8749?product=Firefox&os_name=mac&test_name=Talos%20tp5row
GET /talos/testdata/metrics/Mozilla-Beta/ebfad1bf8749?os_name=mac,linux
GET /talos/testdata/metrics/Mozilla-Beta/ebfad1bf8749?page_name=digg.com,alipay.com,tmall.com
Example response:
Content-Type: application/json
[
{
"pages": {
"icanhascheezburger.com": {
"push_date": 1345523670,
"trend_stddev": 49.8,
"h0_rejected": false,
"p": 0.6,
"fdr": false,
"stddev": 12.3,
"pushlog_id": 6077315,
"trend_mean": 268.5,
"mean": 265.7,
"test_evaluation": true,
"n_replicates": 25
},
"ucoz.ru": {
"push_date": 1345523670,
"trend_stddev": 2.4,
"h0_rejected": false,
"p": 0.6,
"fdr": false,
"stddev": 1.9,
"pushlog_id": 6077315,
"trend_mean": 73.7,
"mean": 73.5,
"test_evaluation": true,
"n_replicates": 25
},
"digg.com": {
"push_date": 1345523670,
"trend_stddev": 3.1,
"h0_rejected": false,
"p": 0.2,
"fdr": false,
"stddev": 3.3,
"pushlog_id": 6077315,
"trend_mean": 151.6,
"mean": 152.2,
"test_evaluation": true,
"n_replicates": 25
},
{ … },
{ … },
{ … },
...
},
"test_machine": {
"platform": "x86",
"osversion": "6.1.7600",
"os": "win",
"name": "talos-r3-w7-060"
},
"testrun": {
"date": 1348091605,
"suite": "Talos tp5row",
"test_run_id": 417
},
"push_info": {
"pushlog_id": 6077315,
"push_date": 1345523670
},
"test_build": {
"name": "Firefox",
"version": "15.0",
"branch": "Mozilla-Beta",
"type": "opt",
"id": "20120820213501",
"revision": "7d3fc54fb002"
}
},
{ … },
{ … },
{ … },
...
]
Return a summary of all test evaluation results for all metrics data associated with the project, branch and revision specified. A test evaluation is a generic representation of test sucess or failure. A test evaluation of false indicates failure and true indicates sucess. All metric methods available implement a generic test evaluation that can be accessed as the metric value test_evaluation.
| Query Parameters: | |
|---|---|
|
|
Example request:
GET /talos/testdata/metrics/Mozilla-Beta/ebfad1bf8749/summary?product=Firefox
GET /talos/testdata/metrics/Mozilla-Beta/ebfad1bf8749/summary?os_name=mac&test_name=Talos tp5row
GET /talos/testdata/metrics/Mozilla-Beta/ebfad1bf8749/summary?os_name=mac,linux
GET /talos/testdata/metrics/Mozilla-Beta/ebfad1bf8749/smmary?page_name=digg.com,alipay.com,tmall.com
Example response:
Content-Type: application/json
{
/* Information about the product data requested */
product_info": {
"version": "15.0",
"name": "Firefox",
"branch": "Mozilla-Beta",
"revision": "7d3fc54fb002"
},
"products": [
/* List of products/branches that are available for this revision */
{
"product": "Firefox",
"version": "15.0",
"branch": "Mozilla-Inbound"
},
{
"product": "Fennec",
"version": "16.0a1",
"branch": "Mozilla-Inbound"
},
{
"product": "Firefox",
"version": "16.0a1",
"branch": "Mozilla-Inbound-Non-PGO"
},
],
/* Push data associated with this revision */
"push_data": {
"node": "1e2b9cdc486b0d23de9d313fdfe24978213d3630",
"branch_id": 65,
"name": "Mozilla-Inbound",
"date": 1340753165,
"user": "someone@mozilla.com",
"pushlog_id": 25823,
"push_id": 11191,
"desc": "Change standards mode height calculations for table cells
to use content-box sizing rather than border-box sizing
by default (and to honor -moz-box-sizing, which we do not
do in quirks mode). Also remove -moz-box-sizing:
border-box from default style for caption element
(all modes). (Bug 248239) r=dbaron"
},
summary": {
"fail": {
"percent": 3,
"value": 25
},
"total_tests": 977,
"pass": {
"percent": 97,
"value": 952
}
},
summary_by_platform": {
"linux fedora 12 x86": {
"fail": {
"percent": 1,
"value": 2
},
"total_tests": 140,
"pass": {
"percent": 99,
"value": 138
}
},
"linux redhat 12 x86": {
"fail": {
"percent": 0,
"value": 0
},
"total_tests": 131,
"pass": {
"percent": 100,
"value": 131
}
},
{ ... },
{ ... },
{ ... },
},
summary_by_test": {
"Talos tp5row": {
"fail": {
"percent": 1,
"value": 9
},
"total_tests": 660,
"pass": {
"percent": 99,
"value": 651
}
},
"Talos tsvg_opacity": {
"fail": {
"percent": 0,
"value": 0
},
"total_tests": 14,
"pass": {
"percent": 100,
"value": 14
}
},
{ ... },
{ ... },
{ ... },
...
},
tests": {
"Talos tp5row": {
"linux fedora 12 x86": {
"fail": {
"percent": 0,
"value": 0
},
"pass": {
"percent": 100,
"value": 91
}
"total_tests": 91,
"pages": [
{ "telegraph.co.uk": true },
{ "web.de": true },
{ "tudou.com": true },
{ "nicovideo.jp": true },
{ "rakuten.co.jp": true },
{ "store.apple.com": true },
{ ... },
...
],
"platform_info": {
"operating_system_version": "fedora 12",
"type": "opt",
"processor": "x86",
"operating_system_name": "linux"
},
},
win 6.1.7600 x86": {
"fail": {
"percent": 7,
"value": 7
},
"pass": {
"percent": 93,
"value": 93
}
"total_tests": 100,
"pages": [
{ "telegraph.co.uk": true },
{ "web.de": true },
{ "tudou.com": true },
{ "nicovideo.jp": false },
{ "rakuten.co.jp": true },
{ "store.apple.com": false },
{ ... },
...
],
"platform_info": {
"operating_system_version": "6.1.7600",
"type": "opt",
"processor": "x86",
"operating_system_name": "win"
},
},
{ ... },
...
},
"Talos tsvg_opacity": { ... },
"Talos tsvg": { ... },
"Talos tdhtml": { ... }
}
Return a pushlog data structure for the given project, branch, revision, test_name, page_name combination decorated with all of the metrics data associated with each push. A number of pushes before and after the target revision can also be specified.
| Query Parameters: | |
|---|---|
|
|
Example request:
GET /talos/testdata/metrics/Mozilla-Inbound/18f7e51126e0/pushlog?product=Firefox&branch_version=16.0a1&os_name=linux&os_version=redhat 12 processor=x86&build_type=opt&test_name=Talos tdhtmlr&page_name=colorfade.html&pushes_before=35&pushes_after=5
GET /talos/testdata/metrics/Mozilla-Inbound/18f7e51126e0/pushlog?product=Firefox&branch_version=16.0a1&os_name=linux&os_version=redhat 12 processor=x86&build_type=opt&test_name=Talos tdhtmlr&page_name=digg.com,alipay.com,tmall.com&pushes_before=35&pushes_after=5
Example response:
Content-Type: application/json
[
{
"branch_name": "Mozilla-Inbound",
"pushlog_id": 6004901,
"metrics_data": [ ... ],
"date": 1345500867,
"dz_revision": "ee34f7b36241",
"push_id": 1303,
"revisions": [
{
"desc": "Fix for bug 768669 (Move remaining DOM list proxy bindings from behind the pref). r=bz.",
"author": "someone <someone@mozilla.org>",
"revision": "cd9ce15216cd"
},
{
"desc": "Fix for bug 768533 (Make mozilla::dom::Uint8ClampedArray::Create call JS_NewUint8ClampedArray). r=bz.",
"author": "someone <someone@mozilla.org>",
"revision": "7c0a1c9ab380"
},
{
"desc": "Fix for bug 768050 (Make TypedArray::Create take a wrapper cache and create JS objects in the compartment of the cache's wrapper). r=bz.",
"author": "someone <someone@mozilla.org>",
"revision": "ee34f7b36241"
}
]
},
{
"branch_name": "Mozilla-Inbound",
"pushlog_id": 6034235,
"metrics_data": [
{
"pages": {
"icanhascheezburger.com": {
"push_date": 1345510991,
"trend_stddev": 20.2,
"h0_rejected": false,
"p": 0.4,
"fdr": false,
"stddev": 23.7,
"pushlog_id": 6034235,
"trend_mean": 237.9,
"mean": 238.8,
"test_evaluation": true,
"n_replicates": 25
},
"ucoz.ru": {
"push_date": 1345510991,
"trend_stddev": 3.1,
"h0_rejected": false,
"p": 0.8,
"fdr": false,
"stddev": 1.6,
"pushlog_id": 6034235,
"trend_mean": 68.4,
"mean": 67.9,
"test_evaluation": true,
"n_replicates": 25
},
{ ... },
{ ... },
{ ... },
},
"test_machine": {
"platform": "x86",
"osversion": "5.1.2600",
"os": "win",
"name": "talos-r3-xp-053"
},
"testrun": {
"date": 1348091605,
"suite": "Talos tp5row",
"test_run_id": 289
},
"push_info": {
"pushlog_id": 6034235,
"push_date": 1345510991
},
"test_build": {
"name": "Firefox",
"version": "15.0",
"branch": "Mozilla-Beta",
"type": "opt",
"id": "20120820180351",
"revision": "b691cd9c10dd"
}
},
{ ... },
{ ... },
{ ... },
],
"date": 1345510991,
"dz_revision": "b691cd9c10dd",
"push_id": 1304,
"revisions": [
{ ... },
{ ... },
{ ... },
]
},
{ ... },
{ ... },
{ ... },
]
},
These are a set of web service endpoints for getting reference data about a Datazilla project.
Return a count of all objectstore entries that have an error. The return value is broken down by two types:
- JSON parse errors
- All other errors
| Query Parameters: | |
|---|---|
|
|
Example request:
GET /talos/refdata/objectstore/error_count?days_ago=10
Example response:
Content-Type: application/json
[
{
"count(id)": 36,
"message": "Malformed JSON"
},
{
"count(id)": 4,
"message": "Other"
}
]
Return a list of all objectstore entries for this project that have an error.
| Query Parameters: | |
|---|---|
|
|
Example request:
GET /talos/refdata/objectstore/error_list?days_ago=10
Example response:
Content-Type: application/json
[
{
"date_loaded": 1343793738,
"id": 127661,
"test_run_id": null,
"worker_id": null,
"processed_flag": "ready",
"error_msg": "Malformed JSON: Expecting , delimiter: line 1 column 52606 (char 52606)"
},
{
"date_loaded": 1343795847,
"id": 127678,
"test_run_id": null,
"worker_id": null,
"processed_flag": "ready",
"error_msg": "Malformed JSON: Expecting , delimiter: line 1 column 51298 (char 51298)"
},
...
]
Return the full JSON blob for id and associated object reference data. It’s possible that the JSON is in a non-parseable state, in which case it will be returned as an escaped string. So the information you’re looking for may require you to dig into the malformed formed JSON without a parser.
Example request:
GET /talos/refdata/objectstore/json_blob/12845
Example response:
Content-Type: application/json
{
"json_blob": {
"test_machine": {
"platform": "x86",
"osversion": "redhat 12",
"os": "linux",
"name": "talos-r3-fed-003"
},
"testrun": {
"date": 1351308688,
"suite": "Talos dromaeo_css",
"options": { … }
},
"results": { … },
"test_build": {
"id": "20121026192237",
"version": "19.0a1",
"name": "Firefox",
"branch": "Mozilla-Inbound-Non-PGO",
"revision": "ba9fb2ed910c"
}
},
"date_loaded": 1355269377,
"error_flag": "N",
"test_run_id": null,
"processed_flag": "ready",
"error_msg": ""
}
Return size (in MegaBytes) of the objectstore database for this project.
Example request:
GET /talos/refdata/objectstore/db_size
Example response:
Content-Type: application/json
[
{
"size_mb": "1740.55",
"db_name": "talos_objectstore_1"
}
]
Return a list of test runs broken down by branch.
| Query Parameters: | |
|---|---|
|
|
Example request:
GET /talos/refdata/perftest/runs_by_branch?days_ago=5
Example response:
Content-Type: application/json
{
"Mozilla-Beta": {
"count": 749
}
Mozilla-Beta-Release-Non-PGO": {
"count": 510,
}
}
Example request:
GET /talos/refdata/perftest/runs_by_branch?days_ago=5&show_test_runs=true
Example response:
Content-Type: application/json
{
"Mozilla-Beta": {
"count": 749,
"test_runs": [
{
"build_id": 2051,
"status": 1,
"date_run": 1344714939,
"test_id": 3,
"product": "Firefox",
"version": "15.0",
"branch": "Mozilla-Beta",
"machine_id": 555,
"id": 132895,
"revision": "50f5c2689179"
},
...
]
}
}
Return a raw list of data from the data_type provided. Valid data_type values are: machines, operating_systems, options, tests, pages, products
Example request:
GET /talos/refdata/perftest/ref_data/operating_systems
Example response:
Content-Type: application/json
{
"macOS X 10.5.8": 5,
"win6.1.7600": 8,
"linuxfedora 12": 4,
...
}
Return size (in MegaBytes) of the perftest database for this project.
Example request:
GET /talos/refdata/perftest/db_size
Example response:
Content-Type: application/json
[
{
size_mb": "10289.78",
"db_name": "talos_perftest_1"
}
]
Return a list of pushlog entries that are not reflected in the perftest data for project.
| Query Parameters: | |
|---|---|
|
|
Example request:
GET /talos/refdata/pushlog/not_referenced/?days_ago=100&branches=Mozilla-Inbound
Example response:
Content-Type: application/json
{
"with_matching_test_run": {
"Mozilla-Inbound": {
"pushlogs": [
{
"push_id": 11171,
"revisions": [
"b4d033913a03",
"85d44a26763c",
"551ad0863475"
]
},
...
]
}
},
"without_matching_test_run": {
"Mozilla-Inbound": {
"pushlogs": [
{
"push_id": 11078,
"revisions": [
"d592966ede4f"
]
},
...
]
}
}
}
Return a list of pushlog entries.
| Query Parameters: | |
|---|---|
|
|
Example request:
GET /refdata/pushlog/list/?days_ago=1&branches=Mozilla-Inbound
Example response:
Content-Type: application/json
{
"14470": {
"branch_name": "Mozilla-Inbound",
"revisions": [
"41cf3c361d9d"
]
},
"14471": {
"branch_name": "Mozilla-Inbound",
"revisions": [
"fd4d9c386f97",
"8a11353cad22",
"a027c9d63d20",
"cb3dd01ba9be",
"14ac87e7546b",
"aa4ba0fc1f8d",
"1cc49d5dcff4",
"c6768c151b64"
]
}
}
Return the list of known pushlog branches.
Example request:
GET /refdata/pushlog/branches
Example response:
Content-Type: application/json
[
"Firefox",
"Mozilla-Inbound",
...
]
Return size (in MegaBytes) of the pushlog database for this project.
Example request:
GET /refdata/pushlog/db_size
Example response:
Content-Type: application/json
[
{
"size_mb": "29.30",
"db_name": "pushlog_hgmozilla_1"
}
]
The following method loads a JSON structure into the objectstore database. This web service method implements two-legged OAuth and requires a consumer key and secret to be supplied in the POST. Each objectstore database in datazilla has an OAuth consumer key and secret associated with it. The values for the OAuth credentials can be found in the datazilla schema. The best way to load data into datazilla is by using the datazilla client which calls this method but also provides helpful methods for building the required JSON structure and implementing the OAuth consumer. A sample JSON structure that datazilla accepts can be found here.
Returns a JSON object
| Query Parameters: | |
|---|---|
|
|
Example response:
Content-Type: application/json
{
"status": "well-formed JSON stored",
"size": 1500,
"url": "https://datazilla.mozilla.com/talos/refdata/objectstore/json_blob/1000"
}
Install memcached, MySQL, a WSGI application server, and a static files web server (in some cases, e.g. Apache, the latter two could be the same server).
Install the compiled dependencies in requirements/compiled.txt via system package manager or via:
pip install -r requirements/compiled.txt
Configure the WSGI application server to serve the application object in datazilla/wsgi.py at the root of the domain, and configure the static files server to serve the files in datazilla/webapp/static at the URL path /static/. See the sample config files for Apache and nginx in datazilla/webapp/sample_configs/.
Copy datazilla/settings/local.sample.py to datazilla/settings/local.py and edit the settings it contains to the correct values for your installation. DATAZILLA_MEMCACHED should be a string like 127.0.0.1:11211 - the host and port at which memcached is running.
Run python manage.py syncdb to create the core datasource table.
Create a paired objectstore and performance test database with the manage.py create_perftest_project command:
manage.py create_perftest_project [options]
Create all databases for a new project.
Options:
-p PROJECT, --project=PROJECT
Set the project to run on: talos, b2g, schema, test
etc....
--perftest_host=PERFTEST_HOST
The host name for the perftest database
--objectstore_host=OBJECTSTORE_HOST
The host name for the objectstore database
--perftest_type=PERFTEST_TYPE
The database type (e.g. 'MySQL-InnoDB') for the
perftest database
--objectstore_type=OBJECTSTORE_TYPE
The database type (e.g. 'MySQL-Aria') for the
objectstore database
--cron_batch=CRON_BATCH
Add this new project to this cron_batch. This value
indicates the size of the project and may determine
how much time between intervals should be set. Larger
projects will likely have a longer time interval
between execution as cron jobs.Choices are: small,
medium, large. Default to None.
Install crontab.txt
Developing datazilla requires installing the dev-only dependencies:
pip install -r requirements/dev.txt
To run the Python tests, run ./runtests.sh (your DATAZILLA_DATABASE_* settings in datazilla/settings/local.py must be configured).
This will output test coverage metrics in HTML format to the htmlcov/ directory.
To add or change a pure-Python production dependency, add or modify the appropriate line in requirements/pure.txt, then run bin/generate-vendor-lib.py. You should see the actual code changes in the dependency reflected in vendor/ if you git diff. Commit both the change to requirements/pure.txt and the changes in vendor/.
To add or change a non-pure-Python production dependency, simply add or modify the appropriate line in requirements/compiled.txt.
To add or change a development-only dependency, simply add or modify the appropriate line in requirements/dev.txt.
There are four database schemas available in datazilla: datazilla, schema_hgmozilla.sql.tmpl, schema_objectstore.sql.tmpl, and schema_perftest.sql.tmpl. Three of these are template schemas that are used by manage commands to create new databases with the storage engine specified by the user. Access to each schema is provided through the Model layer. The model layer is used by controllers to retrieve data in each of the schemas and is exposed to the user through a set of web service methods.
This schema is accessed using the django ORM, the model for it is defined here. and consists of a single table with the following structure.
| Field | Type | Null | Key | Default | Extra |
|---|---|---|---|---|---|
| id | int(11) | NO | PRI | NULL | auto_increment |
| project | varchar(25) | NO | MUL | NULL | |
| contenttype | varchar(25) | NO | NULL | ||
| dataset | int(11) | NO | NULL | ||
| host | varchar(128) | NO | NULL | ||
| read_only_host | varchar(128) | YES | NULL | ||
| name | varchar(128) | NO | NULL | ||
| type | varchar(25) | NO | NULL | ||
| oauth_consumer_key | varchar(45) | YES | NULL | ||
| oauth_consumer_secret | varchar(45) | YES | NULL | ||
| creation_date | datetime | NO | NULL | ||
| cron_batch | varchar(45) | YES | small |
All databases storing data used by datazilla are stored as a row in this table. Each database has three classifiers associated with it: project, contenttype, and dataset. The name of the database is typically these three classifiers joined on an underscore but there is no requirement for this, the name can be any string. There is no physical requirement for the databases referenced in this table to be co-located. The only requirement is that both the web service and machine that run’s the cron jobs have access to each of the databases in this table. Any database can have OAuth credentials associated with it but they are not required so the field can be null. Currently the only databases that require OAuth are the objectstore and only for the storage of the JSON object. Each database can also have a cron batch interval associated with it. This interval specifies the time interval of cron jobs run.
project - A descriptive string associated with the project: talos, b2g, schema etc... This string becomes the location field in the url for related web service methods.
contenttype - A string describing the content type associated with the database. The perftest content type stores performance test results, the objectstore content type stores JSON objects in it. A project can have any number of contenttypes associated with it.
dataset - An integer that can be enumerated. This allows more than one database to exist for the same project/contenttype pair.
host - Name of the database host.
read_only_host - A read only host associated with the database.
name - Name of the database.
type - Type of storage engine associated with the database. This is automatically added to the template schema when a user runs a manage command that creates a database schema. There is currently support for MariaDB and MySQL storage engines.
oauth_consumer_key - The OAuth consumer key. This is created for databases with objectstores automatically by the create_project manage command.
oauth_consumer_secret - The OAuth consumer secret. This is created for databases with objectstores automatically by the create_project manage command.
creation_date - Date the database was created.
cron_batch - The cron interval to use when running cron jobs on this database.
The hgmozilla schema currently holds the mozilla mercurial push log data. However, the only part of it that’s specific to mercurial is the web service method used to retrieve data to populate it. The data used to populate the schema is generated by the json-pushes web service method. The manage command, update_pushlog, calls this web service method and populates the associated schema. The data can be used to create an ordered list of code base changes pushed to the build/test system. This is required for any statistical method that requires a comparison between a push and its parent.
The objectstore schema holds the unprocessed json objects submitted to the project. When objects are successfully processed into a corresponding index the test_run_id field is populated with an integer. The test_run_id corresponds to the test_run.id field in the perftest schema.
This perftest schema translates the JSON structure in the objectstore into a relational index. It also contains tables for the storage of statistical data generated post object submission.
The model layer found in /datazilla/model provides an interface for getting/setting data in a database. The datazilla model classes rely on a module called datasource. This module encapsulates SQL manipulation. All of the SQL used by the system is stored in JSON files found in /datazilla/model/sql. There can be any number of SQL files stored in this format. The JSON structure allows SQL to be stored in named associative arrays that also contain the host type to be associated with each statement. Any command line script or web service method that requires data should use a derived model class to obtain it.
ptm = PerformanceTestModel(project)
products = ptm.get_product_test_os_map()
The ptm.get_product_test_os_map() method looks like this:
def get_product_test_os_map(self):
proc = 'perftest.selects.get_product_test_os_map'
product_tuple = self.sources["perftest"].dhub.execute(
proc=proc,
debug_show=self.DEBUG,
return_type='tuple',
)
return product_tuple
perftest.selects.get_product_test_os_map found in datazilla/model/sql/perftest.json looks like this:
{
"selects":{
"get_product_test_os_map":{
"sql":"SELECT b.product_id, tr.test_id, b.operating_system_id
FROM test_run AS tr
LEFT JOIN build AS b ON tr.build_id = b.id
WHERE b.product_id IN (
SELECT product_id
FROM product )
GROUP BY b.product_id, tr.test_id, b.operating_system_id",
"host":"master_host"
},
"...more SQL statements..."
}
The string, perftest, in perftest.selects.get_product_test_os_map refers to the SQL file name to load in /datazilla/model/sql. The SQL in perftest.json can also be written with placeholders and a string replacement system, see datasource for all of the features available.