> ## Documentation Index
> Fetch the complete documentation index at: https://knowledge.cloudquant.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Queries & Working with Large Datasets

> Best practices for querying large datasets with the CloudQuant Data Liberator API, including point-in-time and time series query patterns.

# Queries & working with large datasets

The CloudQuant Data Liberator service frequently returns extensive result sets, sometimes reaching millions of rows. These queries can be time-consuming to execute. Since data frequencies vary across datasets, it is recommended to start with narrow timeframes for specific symbols before expanding scope.

<Tip>
  Consider running a point-in-time query for one symbol to get an idea of how large your dataset is prior to running other queries.
</Tip>

<Note>
  As of release version 2.0, the P12 client certificate (`liberator.pfx`) is no longer required. The `--cert-type P12 --cert liberator.pfx` flags shown in the curl examples below are only needed for releases prior to 2.0.
</Note>

## Query types

Most dataset queries fall into two categories:

* **Point-in-time queries**
* **Time series queries**

Familiarity with both query types early on will improve your API usage efficiency.

## Point-in-time queries

Omit the `back_to` parameter to receive single point-in-time data for each specified symbol based on the `as_of` date/time. If you also exclude `as_of`, CloudQuant Data Liberator defaults to the current date/time.

<CodeGroup>
  ```python Python theme={null}
  df = liberator.get_dataframe(
      liberator.query(
          name='daily_bars',
          as_of='2020-11-15',
          symbols=['AAPL']
      )
  )
  ```

  ```csharp C# theme={null}
  var res = liberator.query(new Dictionary<string, dynamic>() {
      {"name", "daily_bars"},
      {"as_of", "2020-11-15"},
      {"symbols", "AAPL"}
  });
  ```

  ```javascript JavaScript theme={null}
  let params = {
      symbols: "AAPL",
      name:    "daily_bars",
      as_of:   "2020-11-15"
  };

  liberator.query(params).then(function(results) {
      // Your code
  });
  ```

  ```java Java theme={null}
  Object query_id = liberator.query(new HashMap<>() {{
      put("name", "daily_bars");
      put("as_of", "2020-11-15");
      put("symbols", "AAPL");
  }});
  ```

  ```r R theme={null}
  print(liberator::get_dataframe(liberator::query(
      name="daily_bars",
      as_of="2020-11-15",
      symbols="AAPL"
  )))
  ```

  ```bash RESTful (curl) theme={null}
  curl -qs --cert-type P12 --cert liberator.pfx \
    -H "Content-Type: application/json" --noproxy '*' \
    --data '{"json_xfer":true,"symbols":"AAPL","name":"daily_bars","as_of":"2020-11-15","user":"'"$USER"'","token":"'"$TOKEN"'","system":"API"}' \
    $URL/liberator/query
  ```

  ```cpp C++ theme={null}
  auto table = liberator.get_table(liberator.query({
      {"symbols", std::vector<std::string>{"AAPL"}},
      {"name", "daily_bars"s},
      {"as_of", "2020-11-15"s}
  }));
  ```
</CodeGroup>

## Time series queries

Include a `back_to` parameter that precedes your `as_of` value. When `as_of` is excluded, it defaults to the present moment.

<CodeGroup>
  ```python Python theme={null}
  df = liberator.get_dataframe(
      liberator.query(
          name='daily_bars',
          as_of='2020-11-15',
          back_to='2018-11-15',
          symbols=['FB', 'AAPL', 'NFLX', 'GOOG', 'MSFT', 'IBM']
      )
  )
  ```

  ```csharp C# theme={null}
  var res = liberator.query(new Dictionary<string, dynamic>() {
      {"name", "daily_bars"},
      {"as_of", "2021-04-10"},
      {"back_to", "2021-03-10"},
      {"symbols", "ES"}
  });

  foreach (RecordBatch batch in res())
  {
      var df = DataFrame.FromArrowRecordBatch(batch);
      Console.WriteLine(DataFrameUtils.PrettyPrint(df));
  }
  ```

  ```javascript JavaScript theme={null}
  let params = {
      back_to: "2021-01-12 00:00:00",
      as_of:   "2021-01-12 23:59:59",
      symbols: "AAPL",
      name:    "daily_bars"
  };

  liberator.query(params).then(function(results) {
      // Your code
  });
  ```

  ```java Java theme={null}
  Object query_id = liberator.query(new HashMap<>() {{
      put("name", "daily_bars");
      put("as_of", "2020-11-15");
      put("back_to", "2020-01-01");
      put("symbols", "AAPL");
  }});
  ```

  ```r R theme={null}
  print(liberator::get_dataframe(liberator::query(
      name="daily_bars",
      symbols="AAPL",
      as_of="2019-09-15",
      back_to="2019-09-10"
  )))
  ```

  ```bash RESTful (curl) theme={null}
  curl -qs --cert-type P12 --cert liberator.pfx \
    -H "Content-Type: application/json" --noproxy '*' \
    --data '{"json_xfer":true,"symbols":"AAPL","name":"daily_bars","back_to":"2022-12-01","user":"'"$USER"'","token":"'"$TOKEN"'","system":"API"}' \
    $URL/liberator/query
  ```

  ```cpp C++ theme={null}
  auto table = liberator.get_table(liberator.query({
      {"symbols", std::vector<std::string>{"AAPL", "GOOGL"}},
      {"name", "daily_bars"s},
      {"as_of", "2021-04-10"s},
      {"back_to", "2021-03-10"s}
  }));
  ```
</CodeGroup>

<Warning>
  When using `back_to` or `as_of` parameters, the time component is always used even if you do not specify it. Therefore, if you say `as_of: "2023-01-15"`, you are actually saying `as_of: "2023-01-15 00:00:00"`. This may affect result precision depending on your data requirements.
</Warning>