biothings.tests

biothings.tests.hub

class biothings.tests.hub.DatabaseCollectionTesting(db_url, db, collection)

Bases: object

Constructor that takes in three items db_url - string - the mongoDB url to connect to db - string - name of DB to use collection - string - name of collection in db

test_database_index()

test_document_name()

test_documents_taxid(taxid)

test_field_does_not_exist(_id)

test_field_taxid(taxid)

test_field_unique_id(_id)

test_total_document_count(expected_count)

biothings.tests.web

Biothings Test Helpers

There are two types of test classes that provide utilities to three types of test cases, developed in the standalone apps.

The two types of test classes are:

BiothingsWebTest, which targets a running web server. BiothingsWebAppTest, which targets a web server config file.

To further illustrate, for any biothings web applications, it typically conforms to the following architectures:

Layer 3: A web server that implements the behaviors defined below. Layer 2: A config file that defines how to serve data from ES. Layer 1: An Elasticsearch server with data.

And for the two types of test classes, to explain their differences in the context of the layered design described above:

BiothingsWebTest targets an existing Layer 3 endpoint. BiothingsWebAppTest targets layer 2 and runs its own layer 3. Note no utility is provided to directly talk to layer 1.

The above discussed the python structures provided as programming utilities, on the other hand, there are three types of use cases, or testing objectives:

L3 Data test, which is aimed to test the data integrity of an API.

It subclasses BiothingsWebTest and ensures all layers working. The data has to reside in elasticsearch already.

L3 Feature test, which is aimed to test the API implementation.

It makes sure the settings in config file is reflected. These tests work on production data and require constant updates to keep the test cases in sync with the actual data. These test cases subclass BiothingsWebTest as well and asl require existing production data in elasticsearch.

L2 Feature test, doing basically the same things as above but uses

a small set of data that it ingests into elasticsearch. This is a lightweight test for development and automated testings for each new commit. It comes with data it will ingest in ES and does not require any existing data setup to run.

To illustrate the differences in a chart: +————–+———————+—————–+————-+—————————+ | Objectives | Class | Test Target | ES Has Data | Automated Testing Trigger | +————–+———————+—————–+————-+—————————+ | L3 Data Test | BiothingsWebTest | A Running API | Yes | Data Release | +————–+———————+—————–+————-+—————————+ | L3 Feature T.| BiothingsWebTest | A Running API | Yes | Data Release & New Commit | +————–+———————+—————–+————-+—————————+ | L2 Feature T.| BiothingsWebAppTest | A config module | No* | New Commit | +————–+———————+—————–+————-+—————————+ * For L2 Feature Test, data is defined in the test cases and will be automatically ingested into

Elasticsearch at the start of the testing and get deleted after testing finishes. The other two types of testing require existing production data on the corresponding ES servers.

In development, it is certainly possible for a particular test case to fall under multiple test types, then the developer can use proper inheritance structures to avoid repeating the specific test case.

In terms of naming conventions, sometimes the L3 tests are grouped together and called remote tests, as they mostly target remote servers. And the L2 tests are called local tests, as they starts a local server.

L3 Envs:

TEST_SCHEME TEST_PREFIX TEST_HOST TEST_CONF

L2 Envs:

TEST_KEEPDATA < Config Module Override >

biothings.tests.web.BiothingsDataTest

alias of BiothingsWebTest

biothings.tests.web.BiothingsTestCase

alias of BiothingsWebAppTest

class biothings.tests.web.BiothingsWebAppTest(methodName: str = 'runTest')

Bases: BiothingsWebTestBase, AsyncHTTPTestCase

Starts the tornado application to run tests locally. Need a config.py under the test class folder.

TEST_DATA_DIR_NAME: str | None = None

property config

get_app()

Should be overridden by subclasses to return a tornado.web.Application or other .HTTPServer callback.

get_new_ioloop()

Returns the .IOLoop to use for this test.

By default, a new .IOLoop is created for each test. Subclasses may override this method to return .IOLoop.current() if it is not appropriate to use a new .IOLoop in each tests (for example, if there are global singletons using the default .IOLoop) or if a per-test event loop is being provided by another system (such as pytest-asyncio).

get_url(path)

Try best effort to get a full url to make a request. Return an absolute url when class var ‘host’ is defined. If not, return a path relative to the host root.

request(path, method='GET', expect=200, **kwargs)

Use requests library to make an HTTP request. Ensure path is translated to an absolute path. Conveniently check if status code is as expected.

class biothings.tests.web.BiothingsWebTest

Bases: BiothingsWebTestBase

classmethod setup_class()

this is the setup method when pytest run tests from this class

class biothings.tests.web.BiothingsWebTestBase

Bases: object

get_url(path)

Try best effort to get a full url to make a request. Return an absolute url when class var ‘host’ is defined. If not, return a path relative to the host root.

host = ''

static msgpack_ok(packed_bytes)

Load msgpack into a dict

prefix = 'v1'

query(method='GET', endpoint='query', hits=True, data=None, json=None, **kwargs)

Make a query and assert positive hits by default. Assert zero hit when hits is set to False.

request(path, method='GET', expect=200, **kwargs)

Use requests library to make an HTTP request. Ensure path is translated to an absolute path. Conveniently check if status code is as expected.

scheme = 'http'

static value_in_result(value, result: dict | list, key: str, case_insensitive: bool = False) → bool

Check if value is in result at specific key

Elasticsearch does not care if a field has one or more values (arrays), so you may get a search with multiple values in one field. You were expecting a result of type T but now you have a List[T] which is bad. In testing, usually any one element in the list eq. to the value you’re looking for, you don’t really care which. This helper function checks if the value is at a key, regardless of the details of nesting, so you can just do this:

assert self.value_in_result(value, result, ‘where.it.should.be’)

Caveats: case_insensitive only calls .lower() and does not care about locale/ unicode/anything

Parameters:

• value – value to look for

• result – dict or list of input, most likely from the APIs

• key – dot delimited key notation

• case_insensitive – for str comparisons, invoke .lower() first

Returns:

boolean indicating whether the value is found at the key

Raises:

TypeError – when case_insensitive set to true on unsupported types