Working with the Lookups Table

This lookups feature (System | Saved Objects | Lookups) allows you to centrally create and manage lists of IP addresses, text strings, dates, or numbers for conditional matching in queries and alert filters that you create and edit in any of the query and alert filter builder tools such as:

This is particularly helpful if you want to reuse values for conditions in multiple queries and to import match values in bulk.

Use of this feature requires the Lookup privilege.

The table has common behaviors to all tables in Stellar Cyber, including column management, sorting, editing, or deleting. The Export CSV button at the top of the table is used to export the whole table as a list. Each row, however, includes a button that allows you to Download CSV formatted list for archive and editing offline. You can edit the list and reimport it, or edit it manually.

This feature is for reusable content in query builders and alert filter builders.

Creating or Editing a Lookup List

Create your list with the intention of matching against a specific field such as app_id. Use the instructions in this procedure to create a new list, edit list entries, or upload a replacement set of values to an existing list.

Note the following points:

  • Changes you make to an existing list affect all queries referring to the list.

  • You cannot change the tenant for a list that is in use.

  • If you want to use a lookup list for different tenants, either set it to All Tenants, or create the list again for each individual tenant.

The scope of a query determines which tenants can use it. Whether it's possible to create a query for All Tenants depends on the scope of the object being queried, such as charts, correlations, and Automated Threat-Hunting (ATH) rules. In short, the scope of a queried object cannot be more restrictive than the scope of the query itself. For example, if you create an ATH rule for All Tenants, then the query for this ATH rule can either be All Tenants or just a single tenant, such as "Tenant A" for example. However, if you create an ATH rule for Tenant A, then the query cannot be for All Tenants because the other tenants won’t have this ATH rule and won’t be able to query it. In this case, the query can only be for Tenant A.

  1. To create a new lookup list, select the + Create button.

    2. To edit a list, select the  Edit this row icon near the far right of a lookups list row.

  2. In the General section, enter or edit a Name for the lookup list, and for a new list, set the Tenant and the Type of items in the list.

    The Name appears in the Query Builder Value menu when the Operator is either is in lookup or is not in lookup.

    Special characters are not permitted in name fields for Queries, Lookup lists, or Reports/Dashboards. Letters, underscores, spaces, dashes, numbers and periods are permitted.

    For the Tenant, choose either All Tenants so that the list is available for all the tenants in your system or choose an individual tenant so that the list is available only for it.

    The Type indicates the format of the values that the list contains: string, number, IP, or date (see Syntax, below).

    Screen capture of the General section of the Lookup List configuration

  3. Select Next to advance to the List Definition section.

    You have the option to import a list in CSV format or add individual list items manually.

  4. To import a list of items in a CSV file, select Import from CSV and upload the file (see File Format, below).

    or

    To add individual items manually, enter the first item in the field, and then select the blue Add button to create a new field and add the item next item there. Continue adding fields and populating them with items until your list is complete.

    When the import completes or you've added all the items manually, the List Definition section displays all the items, one per field, as well as the total number.

    Screen capture of the List Definition section of the Lookup List configuration

    Some notes about importing a CSV file:

    • If you are editing rather than creating a new list, imported records fully replace the existing list.

    • Changes you make to an existing list affect all queries referring to the list.

    • The count of imported records might differ from the number of entries in your import file because the import retrieves only entries that match the specified file type.

  5. Select Next to advance to the Review section and check that the name, tenant, and total number of items are correct and that the configuration is free of error (see Limits and Validation, below).

    Screen capture of the Review section of a lookup list configuration

  6. When satisfied, select Submit to save the lookup list.

Syntax

All entries for a list must be of the same type.

  • String: Alphanumeric and symbols

    If you are supplying a list of hashes to use in queries, do not include the SHA= or MD5= prefix. Just include the hash value itself.

  • Number: Integer and decimal.

  • IP address: IPv4 and IPv6 format supported. CIDR format is supported (for example, 192.0.1.0/24).

  • Date: Unix Epoch or ISO 8601 date formats

    Examples:

    Epoch (in milliseconds is recommended unless you know the field you are matching specifically uses another format)

    1641852524004

    Date only forms (variations of YYYY-MM-DD)

    2021-12

    2021-10-25

    Date plus time forms (variations of above plus time):

    1995-02-04T24:00

File Format

Use the following notes as a guide when creating or editing your import file:

  • Import files must have a .csv extension.

  • The content should be in a single column list, one entry per row.

    If your import file includes more than one column, only the first column is used. For example, given the CSV file entries below:

    ssh, 22
http, 8080, 8443
ubuntu, 631, 5298, 5900

    The imported list elements are: ssh, http, and ubuntu

Limits and Validation

Stellar Cyber performs the following checks when you are creating or editing a list:

  • A maximum of 1K entries are allowed per list. For efficiency, however, smaller list sizes are strongly recommended.

  • Duplicate values are noted and not automatically removed.

  • Syntax is matched to the specified list Type. Errors are noted for manual correction.

  • Numbers and symbols are permitted as String elements.

Special characters are not permitted in name fields for Queries, Lookup lists, or Reports/Dashboards. Letters, underscores, spaces, dashes, numbers and periods are permitted.

Deleting a Lookup List

When a lookup list is in use, the delete button is not available.

  1. To identify the query that is using the list, hover your cursor over In Use.

    A tooltip displays the query names that refer to the list.

  2. Access the Queries table to locate the named queries and edit them to remove the lookup list.

    When you remove all references to the list, the delete button becomes available.

Exporting a Lookup List

The CSV export at the top of the table is used to export the whole table as a list. Each row, however, includes a Download CSV button that lets you download a list for archive and editing offline. You can edit the list and reimport it, or you can edit it manually in the UI.

Using a Lookup List

Use the lookups when you add or edit a condition within an alert filter or query, such as when you are editing a report.

  1. Select the Operator menu to choose the needed lookup condition.

    This operator is displayed in the menu when you have the Lookup privilege enabled.

    • is in lookup: Use this to match any of the items in the selected list (boolean OR).

    • is not in lookup: Use this to match anything that is not in the selected list (boolean AND NOT).

  2. For the Value select the lookup list Name for use in the rule.