Configuring Google Cloud Audit Log Connectors

Connectors allow Stellar Cyber to collect data from external sources and add it to the data lake.

Stellar Cyber connectors with the Collect function (collectors) may skip collecting some data when the ingestion volume is large, which potentially can lead to data loss. This can happen when the processing capacity of the collector is exceeded.

Connector Overview: Google Cloud Audit Log

Capabilities

  • Collect: Yes

  • Respond: No

  • Native Alerts Mapped: No

  • Runs on: DP

  • Interval: Configurable

Collected Data

Content Type

Index

Locating Records

Admin Activity

Data Access

Policy

Requests

System Event

  Syslog

msg_class:

google_cloud_admin_activity

google_cloud_data_access

google_cloud_policy

google_cloud_requests

google_cloud_system_event

msg_origin.source:

google_cloud

msg_origin.vendor:

google

msg_origin.category:

paas

Domain

N/A

Response Actions

N/A

Third Party Native Alert Integration Details

N/A

Required Credentials

  • To set up the service account for this configuration the Google Cloud user must have the role of serviceAccountKeyAdmin.

  • The following privileges are assigned to the service account for the corresponding log access: Logs Viewer and Private Logs Viewer (for the Data Access logs).

  • Audit Logs for the service account must have Admin Read, Data Read, and Data Write enabled for the Kubernetes Engine API.

               Let us know if you find the above overview useful.

Adding a Google Audit Log Connector

To add a Google Audit Log connector:

  1. Enable the Google Cloud Logging API
  2. Create a Google Cloud service account
  3. Create a JSON key
  4. Enable Kubernetes Log Access
  5. Add the connector in Stellar Cyber
  6. Test the connector
  7. Verify ingestion

Enabling the Google Cloud Logging API

Before you create a service account, ensure you are in the correct project and enable the Admin SDK.

Use our example as a guideline, as you might be using a different software version.

  1. Go to https://console.cloud.google.com .

  2. Using the project menu at the top left of the banner, select the project you want to configure to ensure you set up a service account in the correct project area.

  3. Click to display the menu.

  4. Choose APIs & Services > Library. The API Library page appears.

  5. Search for logging . The Cloud Logging API option should be at the top of the results.

  6. Click Cloud Logging API to display the API detail page.

  7. If it is not already enabled, click Enable.

     

Creating a Google Service Account

With the API enabled, now create a service account.

  1. Navigate back to the main menu of https://console.cloud.google.com.

  2. Click to display the navigation pane.

  3. Choose IAM & Admin > Service Accounts. The service accounts pane for the active project displays.

  4. Verify that you are in the correct project, then click + Create Service Account.

  5. In the Service account details page, enter a Service Account Name in the first field. The Service account ID is automatically created based on the name you supplied.

  6. Enter a description for this service account, to identify its association with your Stellar Cyber connector.

  7. Click the Create and Continue button.

  8. Bypass completing the remaining access control and permission steps, which will be configured in a later step.

    • In the section for granting access to the project, do not specify any roles. Click Continue.

    • In the section for granting users access to the service account, do not specify any users. Click Done.

  9. The new service account is created and displayed on the Service account details page.

  10. Make note of the email address displayed for the service account.

  11. Now give permissions to the Service Account you created. From the IAM & Admin menu, select IAM > Permissions, then click Add.

  12. In the dialog that opens, use the New principals field to search for the email address of the service account name you created above.

  13. Use the Select a role field to search for Logs Viewer, and select that role.

    This role gives read-only access to Admin Activity, Policy Denied, and System Event audit logs. If you have just this role, you cannot view Data Access audit logs.

  14. Click the button to Add Another Role.

  15. Use the new Select a role field to search for Private Logs Viewer, and select that role.

    The Private Logs Viewer role includes the ability to read Data Access audit logs.

  16. Click Save.

  17. The permissions page now displays the service account with the needed permissions to collect logs.

Creating a Service Account Key

With a service account, you can now create a key file for uploading to Stellar Cyber.

  1. Still within the IAM & Admin > Service Accounts page, select the service account you created.

  2. Click in the actions column for that account and select Manage keys from the menu.

    The Keys screen appears.

  3. Click the Add Key menu and select Create new key.

  4. When the dialog displays a dialog Create private key, select JSON.

  5. Click Create. The JSON file is downloaded. Keep this file safe, as you cannot recover the key if this file is lost. You will upload this file to Stellar Cyber when configuring the connector.

  6. Click Close.

Your key file contains information similar to below, which is used by the Stellar Cyber connector.

Copy

Google Audit Logs JSON Key file

    
type    "service_account"
project_id    "audit-gke"
private_key_id    "a00bd1925040fcf0ed46e886c3475f7d149ee2ca"
private_key    "-----BEGIN PRIVATE KEY-----\nMIIEvAIBADANBgk .... clipped out ....EFAASCBKYwggSiAgEAAoIBAQDKFRmHwoCYw8YT\nJuVvwbQJKvtwYXA9ADG0ob9pfVuIB036zttxtjwZi1JKLx/8GEOwx/hdn9hrQvuA\nnGVQRUMMMAUaWZAy0GMTSYH1BUS1dv9KHsCXcprAE2z2Bg+6vkoy16X1h9Rxt5p4\niPf0to8zdUQs1DLr9+C9rxpNDexXizgMMDj2J10qpqI6nGedtAigr+ZPCBoP9Byw\nG3pxakYZx25OzW7lEWr2NBL3nDzivbIqLyW63xvQSywpbuR0IU8tKGhAKSNSnxq8\nrCxqBcvQcvbpvbniNbYMNPuRxfYaFYdeNxga9My0SfwkIY3604y5FNud5+w0kqEY\n2ysGs7ThAgMBAAECggEAAdBsHXh78b8kbxuk0La+nVxUg/gKd0A6KgHPmXB+wNUm\ntHgs3ve+sPCj2ELVetSLj0XAzXOd4GriTc+NnumTxDtGQPQF0fh2Irw4u4q4yOei\ny6M0ebmVkUDNZCUmWQ/8M9oU8TwPJpcSNte9MN4ekYzZfAzqwPy9eKt2uSNNVFT5\nbAiGuWcn3P4EOVeHFjQ0rXn8N6xqg4dtNcJfX1cGvUv3tkwH4Nv+WrO3I0yI7SyF\noV7906wY0M//iuenViUZdJ6+mF6R027DxxUaEF7BE9iwkaOrcGICNJb9oL9xRx8f\nKhpMlxlhg5MAe58/8eBqatz5Oh84bsbAzT59/haygQKBgQD9cfiIYWP22uMLzPgs\n6O4rLjApqdkrWlcbuDGQjCw12LwURQstQ8zSLI7JdTH5l0tWgHNYDcsBNeOLCND8\nltvzD2v9RlyziQqu0yoXScZ0oKE/eTzmnHV8qeEpCGMBrcdu7UMO01/YAITfcbEE\n0EGwKsn50hD+CR2OSLwBpEDKgQKBgQDMHpWeM24mYTib1W4yXYNNbMWEM3M/ETtg\n6byFtj06CNnaXWnnt3zTHg5GaqO1mX3Wgi4Lhx7L+yZ+Ew5qRTX6Z3u8cXPp4Niv\nQk+4wb4NGso75idYzd74cMAnccOmJEE+ZybVh/AQcnDS1f2wgNMxIh4Ubruyrp6U\ncGdJ3+X6YQKBgHHC+6Tz2jQO8kT0vx0hfO6C7/lzlNhh16HInu7cDGgrHdkXWLn7\nqe9ZMsp4wXLgF1//jkajAlFLyBQ7Y9tNN/M1I+ru72NjCxmIDWegpOTqyRY9rDlz\nV0Ua18dcTVyOjWtcMnJwLKA/M2U8mhGoDSQvQLPTN+9fVhXszf8KpE6BAoGAO8mJ\nkXkG/n7jkeGGEYKKfxcpE3vueb6WYeCUs6WZcXI+PVO3PYPlMAFRtRZvNTXsQfSH\ngCDnyF+b+rmiM0ZqdRnyiFPC2/JcBZmquDcnb7c1HS5gwAYtFSARaHxODk/4PEsC\nePEERA0BNwAu+pABTBbuMJXurHuxP1FdL6pBNoECgYASvtL0kAMQcdCYV+qiMSND\nftuiacthYJ3WjpYSCan2xQV18WSX2nsD3206x8oKwC/FguHA9AsFw97PGdVtPUXb\nMrtBCz84cN+j9NPcDV/R1OTKTOHUljONwdNy0ewa6BErWOLAlmZ3Qcn8gmbixPJM\nqNKEDlQFULGuPDazrsJ7/A==\n-----END PRIVATE KEY-----\n"
client_email    "stellarcyber-gke-audit-log@audit-gke.iam.gserviceaccount.com"
client_id    "101041183374632572711"
auth_uri    "https://accounts.google.com/o/oauth2/auth"
token_uri    "https://oauth2.googleapis.com/token"
auth_provider_x509_cert_url    "https://www.googleapis.com/oauth2/v1/certs"
client_x509_cert_url    "https://www.googleapis.com/robot/v1/metadata/x509/stellarcyber-gke-audit-log%40audit-gke.iam.gserviceaccount.com"

Enabling Kubernetes Log Access

To enable access to the logs:

  1. Return to the IAM & Admin menu.

  2. Select the Audit Logs sub-menu.

  3. In the page that displays, locate the filter field for Data Access audit logs configuration.

  4. Search for Kubernetes and select the Kubernetes Engine API.

  5. If the API does not have permissions enabled for Admin Read, Data Read, and Data Write, click the check box to the left of the Service label.

  6. Check the corresponding boxes in the Info Pane.

  7. Click Save.

Adding the Connector in Stellar Cyber

With the Google service account set up and the API key ready, you can add the Google Audit Log connector in Stellar Cyber:

  1. Log in to Stellar Cyber.

  2. Click System | Integration | Connectors. The Connector Overview appears.

  3. Click Create. The General tab of the Add Connector screen appears. The information on this tab cannot be changed after you add the connector.

    The asterisk (*) indicates a required field.

  4. Choose PaaS from the Category drop-down.

  5. Choose Google Cloud Audit Logging from the Type drop-down.

  6. For this connector, the supported Function is Collect, which is enabled already.

  7. Enter a Name.

    Notes:
    • This field does not accept multibyte characters.
    • It is recommended that you follow a naming convention such as tenantname-connectortype.

  8. Choose a Tenant Name. This is the tenant allowed to use the connector.

  9. Choose the device on which to run the connector.

    • Certain connectors can be run on either a Sensor or a Data Processor. The available devices are displayed in the Run On menu. If you want to associate your collector with a sensor, you must have configured that sensor prior to configuring the connector or you will not be able to select it during initial configuration. If you select Data Processor, you will need to associate the connector with a Data Analyzer profile as a separate step. That step is not required for a sensor, which is configured with only one possible profile.

    • If the device you're connecting to is on premises, we recommend you run on the local sensor. If you're connecting to a cloud service, we recommend you run on the DP.

  10. (Optional) When the Function is Collect, you can apply Log Filters. For information, see Managing Log Filters.

  11. Click Next. The Configuration tab appears.

    The asterisk (*) indicates a required field.

  12. Upload the JSON Key File you created above.

  13. Enter the Interval (min). This is how often the logs are collected.

  14. Choose the Content Type you want to collect. The following logs are supported:

    • Admin Activity

    • Data Access

    • Policy

    • Requests

    • System Event

    The Google API requires that the content type you specify here be a type for which you have viewed logs at least once in the corresponding account.

  15. Toggle whether you want to collect All data (default) for the content type or just the GKE (Google Kubernetes Engine) logs.

  16. Click Next. The final confirmation tab appears.

  17. Click Submit.

    To pull data, a connector must be added to a Data Analyzer profile if it is running on the Data Processor.

  18. If you are adding rather than editing a connector with the Collect function enabled and you specified for it to run on a Data Processor, a dialog box now prompts you to add the connector to the default Data Analyzer profile. Click Cancel to leave it out of the default profile or click OK to add it to the default profile.

    • This prompt only occurs during the initial create connector process when Collect is enabled.

    • Certain connectors can be run on either a Sensor or a Data Processor, and some are best run on one versus the other. In any case, when the connector is run on a Data Processor, that connector must be included in a Data Analyzer profile. If you leave it out of the default profile, you must add it to another profile. You need the Administrator Root scope to add the connector to the Data Analyzer profile. If you do not have privileges to configure Data Analyzer profiles, a dialog displays recommending you ask your administrator to add it for you.

    • The first time you add a Collect connector to a profile, it pulls data immediately and then not again until the scheduled interval has elapsed. If the connector configuration dialog did not offer an option to set a specific interval, it is run every five minutes. Exceptions to this default interval are the Proofpoint on Demand (pulls data every 1 hour) and Azure Event Hub (continuously pulls data) connectors. The intervals for each connector are listed in the Connector Types & Functions topic.

    The Connector Overview appears.

The new connector is immediately active.

Testing the Connector

When you add (or edit) a connector, we recommend that you run a test to validate the connectivity parameters you entered. (The test validates only the authentication / connectivity; it does not validate data flow).

  1. Click System | Integrations | Connectors. The Connector Overview appears.

  2. Locate the connector that you added, or modified, or that you want to test.

  3. Click Test at the right side of that row. The test runs immediately.

    Note that you may run only one test at a time.

Stellar Cyber conducts a basic connectivity test for the connector and reports a success or failure result. A successful test indicates that you entered all of the connector information correctly.

To aid troubleshooting your connector, the dialog remains open until you explicitly close it by using the X button. If the test fails, you can select the  button from the same row to review and correct issues.

The connector status is updated every five (5) minutes. A successful test clears the connector status, but if issues persist, the status reverts to failed after a minute.

Repeat the test as needed.

ClosedDisplay sample messages...

Success !

Failure with summary of issue:

Show More example detail:

If the test fails, the common HTTP status error codes are as follows:

HTTP Error Code HTTP Standard Error Name Explanation Recommendation
400 Bad Request This error occurs when there is an error in the connector configuration.

Did you configure the connector correctly?
401 Unauthorized

This error occurs when an authentication credential is invalid or when a user does not have sufficient privileges to access a specific API.

Did you enter your credentials correctly?

Are your credentials expired?

Are your credentials entitled or licensed for that specific resource?
403 Forbidden This error occurs when the permission or scope is not correct in a valid credential.

Did you enter your credentials correctly?

Do you have the required role or permissions for that credential?

404 Not Found This error occurs when a URL path does not resolve to an entity. Did you enter your API URL correctly?
429 Too Many Requests

This error occurs when the API server receives too much traffic or if a user’s license or entitlement quota is exceeded.

The server or user license/quota will eventually recover. The connector will periodically retry the query.

If this occurs unexpectedly or too often, work with your API provider to investigate the server limits, user licensing, or quotas.

For a full list of codes, refer to HTTP response status codes.

The Google API requires that the content type you specify here be a type for which you have viewed logs at least once in the corresponding account. If you run the test for a content type that has not been viewed, the test will result in an error.

Verifying Ingestion

To verify ingestion:

  1. Click Investigate | Threat Hunting. The Interflow Search tab appears.

  2. Change the Indices to Syslog. The table immediately updates to show ingested Interflow records.