> ## Documentation Index
> Fetch the complete documentation index at: https://knowledge.cloudquant.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Advanced - Query Data Using requests.post RESTful API

> How to query CloudQuant Data Liberator data directly using Python's requests.post method with the RESTful API.

# Advanced - query data using requests.post RESTful API

<Note>
  As of release version 2.0, the P12 client certificate (`liberator.pfx`) is no longer required. The `cert=cert` argument and `pfx_to_pem` helper shown below are only needed for releases prior to 2.0.
</Note>

## requests.post from Python to access CloudQuant Data Liberator data

`requests.post` is an HTTP method you can use to send query arguments to the CloudQuant Data Liberator server for processing.

## Post arguments

* **`{base_url}/liberator/query`** — The query endpoint. Use the base URL from your Liberator Profile page download.

* **`data=json.dumps(...)`** - This `json.dumps` section provides the body of the query with these parameters:
  * `compress`
  * `json_xfer`
  * `user`
  * `token`
  * `name`
  * `as_of`
  * `back_to`
  * `symbols`
  * `system`

* **`headers={'Content-Type':'application/json'}`**

* **`cert=cert`** - cert comes from `with pfx_to_pem('liberator.pfx','') as cert:`

## Query the last known value(s)

To get the last known value in any dataset, do not provide the `as_of` or `back_to` arguments. This will give you the last known value for the given dataset and symbols.

## Query for a time series result

Adding the `back_to` argument for any query will give you the time series data all the way back to the specified date.

<Warning>
  Every dataset has different data frequencies. Use short time frames until you become familiar with the datasets. Some datasets are quite large, and an over-extended request can return an extremely large amount of data.
</Warning>

## json.dumps components

| Argument    | Description                                                                                                                      | Type                                             | Example                              |
| ----------- | -------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------ | ------------------------------------ |
| `symbols`   | The security trading symbol(s) you wish to query                                                                                 | String, or List                                  | `"symbols":["AAPL","TLT","GOOG"]`    |
| `name`      | The name of the dataset **(Required)**                                                                                           | String                                           | `"name": "daily_bars"`               |
| `as_of`     | This value can be any past date so that you can see the data as it was known on the "as of" date. Defaults to now.               | String `YYYY-MM-DD HH:MM:SS` (HH:MM:SS optional) | `"as_of":"2020-11-22 19:51:31"`      |
| `back_to`   | The date where the return dataset should begin.                                                                                  | String `YYYY-MM-DD HH:MM:SS` (HH:MM:SS optional) | `"back_to":"2020-01-01"`             |
| `url`       | Optional. The URL of the CloudQuant Data Liberator server. Preconfigured in clients downloaded from your Liberator Profile page. | String                                           | `'http://127.0.0.1:47753'`           |
| `system`    | The name of the authorized system from which you are querying.                                                                   | String                                           | `"system":"API"`                     |
| `compress`  | The data compression method on the wire.                                                                                         | Boolean                                          | `True` or `False`                    |
| `json_xfer` | JSON transfer. Usually False.                                                                                                    | Boolean (Always False)                           | `"json_xfer":False`                  |
| `user`      | The user identifier (as assigned by CloudQuant)                                                                                  | String                                           | `"user":"myUserID"`                  |
| `token`     | The user's assigned token                                                                                                        | String                                           | `"token":"mypersonal-private-token"` |

## Example: calling the CloudQuant Data Liberator REST API with Python

```python theme={null}
import sys
import zlib
import json
import base64
import requests
import tempfile
import contextlib
import pyarrow as pa
import OpenSSL.crypto


@contextlib.contextmanager
def pfx_to_pem(pfx_path, pfx_password):
    '''Decrypts the .pfx file to be used with requests.'''
    with tempfile.NamedTemporaryFile(suffix='.pem') as t_pem:
        t_pem.close()
        f_pem = open(t_pem.name, 'wb')
        pfx = open(pfx_path, 'rb').read()
        p12 = OpenSSL.crypto.load_pkcs12(pfx, pfx_password)
        f_pem.write(OpenSSL.crypto.dump_privatekey(OpenSSL.crypto.FILETYPE_PEM, p12.get_privatekey()))
        f_pem.write(OpenSSL.crypto.dump_certificate(OpenSSL.crypto.FILETYPE_PEM, p12.get_certificate()))
        ca = p12.get_ca_certificates()
        if ca is not None:
            for cert in ca:
                f_pem.write(OpenSSL.crypto.dump_certificate(OpenSSL.crypto.FILETYPE_PEM, cert))
        f_pem.close()
        yield t_pem.name


base_url = '<your-base-url>'  # from Liberator Profile page download
compressed_transfer = True

# POST the query and prepare for a stream of single-line JSON replies
with pfx_to_pem('liberator.pfx', '') as cert:
    r = requests.post(f'{base_url}/liberator/query',
            data=json.dumps({"compress": compressed_transfer, "json_xfer": False,
                "user": user, "token": token,
                "name": name, "as_of": as_of, "back_to": back_to, "symbols": symbols,
                "system": "API"}),
            headers={'Content-Type': 'application/json'},
            cert=cert,
            stream=True)

# Ensure that the request was successful
if r.status_code != 200:
    print(r.json())
else:
    batches = []
    for line in r.iter_lines(4096):
        # Show progress
        print('.', end='')
        sys.stdout.flush()
        # The response stream contains informational messages other than "batch"
        if line[0:14] == b'{"exception":"':
            print('Error: ' + json.loads(''.join([chr(c) for c in line]), encoding='latin-1')['value'])
            break
        if line[0:10] != b'{"batch":"':
            continue
        # Cut out the payload from the JSON line without json module overhead
        decoded = base64.b64decode(line[10:-3])
        # Interpret the payload as an Apache Arrow IPC stream
        reader = pa.ipc.open_stream(zlib.decompress(decoded) if compressed_transfer else decoded)
        # Accumulate RecordBatch objects in a list
        batches.extend([batch for batch in reader])
    print('')
    if batches:
        # Create an Arrow Table view on the batches and convert to a pandas DataFrame
        df = pa.Table.from_batches(batches).to_pandas()
        if 'muts' in df and '_seq' in df:
            # By default, the data is sorted by symbol... re-sort by time
            df = df.sort_values(['muts', '_seq'])
            df.reset_index(drop=True, inplace=True)
        print(df)
```