X
January 22, 2018

Operating OCI Tenancies - Retrieving OCI Audit logs

By: Saurabh Bangad | Cloud Evengelist EMEA

Share

1       {C}Operating OCI Tenancies – Retrieving OCI Audit logs

The OCI Audit service comes complementary with each tenancy which records activity across the tenancy for all supported OCI services. This information can be retrieved and processed by a tenancy owner for various use-cases.

 

In this blog post, I will show how do we retrieve this data for a given time period and the use-cases around its consumption.

2       {C}About OCI Audit Service and use-cases

The Oracle Cloud Infrastructure (OCI) service automatically records calls to all supported Oracle Cloud Infrastructure public API endpoints as log events. This information can be leveraged by customers for the following purposes:

{C}1.     {C}Governance

{C}2.     {C}Industry and regulatory compliance

{C}3.     {C}Operational and/or security processes

{C}4.     {C}Risk Auditing

 

Customers can collect logs centrally using log management or security incident and event management (SIEM) solutions. The following are among common use-cases:

{C}1.     {C}Monitoring – For example, to decrease incident response times.

{C}2.     {C}Security – For example, to identifying suspicious activity across the tenancy mitigate variety of risks, breaches and/or any unauthorized events.

{C}3.     {C}Debugging in Test/Troubleshooting in Production – For example, root cause analysis purposes during incidents.

{C}4.     {C}Analytics – For example, analyzing and identifying patterns for various events occurring in a tenancy.

{C}5.     {C}Machine learning – For example, using this as training data.

 

The OCI Audit service by default is on recording mode and cannot be turned off. The default retention period 90 days, but as a best practice, customers should change the policy to the maximum retention.

3       {C}Retrieving Audit events

In order to make use of audit events, the first step is to retrieve and store audit events. Let’s take a look into the ways in which an Audit event can be retrieved:

{C}o   {C}OCI Console – With OCI user credential, customers can log in to the OCI console to access the Audit service. For example, when customers are trying the OCI Audit service for the first time. This would help them take a first look into handful events.

{C}o   {C}OCI CLI – With OCI CLI customers can make use of the service to retrieve events for a defined compartment for a region specified as per CLI’s config. The OCI CLI command would look like: #oci audit event list --start-time $start-time --end-time $end-time --compartment-id $compartment-id

{C}o   {C}OCI SDKs – With OCI SDKs customers can choose a supported language and retrieve Audit events with the ListEvents API. For production use-cases, this would be the best suitable option.

For this blog post, I will use the OCI Python SDK; I will make use of the <github link>.

3.1      {C}About <Github link> python script

As mentioned above, let’s use the python script, which retrieves logs for a tenancy for a given time perioid across

{C}o   {C}All the regions, including the regions that may be added in future

{C}o   {C}All compartments, including the root compartment

3.1.1    Pre-requisites

Before you make use of this script, it is important to understand its pre-requisites:

{C}1.     {C}Installation of OCI Python SDK – https://oracle-cloud-infrastructure-python-sdk.readthedocs.io/en/latest/installation.html

{C}2.     {C}Configuration of OCI Python SDK – The following are the important parameters for the local configuration:

a.     {C}User OCID

b.     {C}RSA private key in PEM format

c.     {C}Its fingerprint

d.     {C}Tenancy OCID

For additional details check https://docs.us-phoenix-1.oraclecloud.com/Content/API/Concepts/sdkconfig.htm

{C}3.     {C}Add the public key into the OCI IAM user.

{C}4.     {C}OCI IAM Permissions – For the user OCID used above to execute the python script, you need to have the following permissions:

a.     {C}READ permissions for audit-events.

b.     {C}The policy would look like: Allow group <GroupName> to READ audit-events in tenancy

3.1.2    Recommendations when using the script

While using this script, the following are few recommendations:

{C}1.     {C}The script may take few minutes to complete execution. The time it takes to execute the script is directly dependent on the following:

    1. Number of compartments under the tenancy
    2. Number of resources for each compartment for each region
    3. Number of updates to their OCI environment (e.g. start/stop/termination of instances) – Write events
    4. The method in which you regularly query their OCI metadata (e.g. using console vs CLI) – Read events
    5. The time span for which the Audit Events are queried (e.g. 20 minutes, 1 hr, 365days)

{C}2.     {C}This script works on a single tenancy, so customers should take necessary steps to run the script across all their existing tenancies.

{C}3.     {C}In order to collect all the logs, the script needs to be executed at a regular interval.

{C}4.     {C}The results should be stored and may be processed for downstream SIEM tools before further consumption.

{C}5.     {C}Monitor the script results on a regular basis for any exceptions that may be produced.

4       {C}Example use-case and analysis of events

Let’s focus on a use-case where I would like to find out how many changes did I have to my tenancy across a month. For this use-case, I would have to retrieve all the audit events and filter write operations such as Create/Modify/Delete operation on OCI resources, which can be done after eliminating all the list/get operations required to retrieve the metadata.

 

For my test tenancy, consisting of 13 compartments, Audit entries were retrieved for a timespan of a month. The following were the timestamps:

start_time=‘2017-11-05T00:00:00 GMT’ to end_time=‘2017-12-05T00:00:00GMT’

 

The script took about 6 minutes to execute fully on a VM.Standard1.8 with Ubuntu 16.04.

 

The following is a table that shows number of write events vs all the events throughout my tenancy.

Header

Only write events

All the events

Number of events

57

32,713

Number of lines in the logs

5,647

2,961,020

Size of the log

388KB

185MB

5       {C}Conclusion

In this blog, we learned about OCI Audit Service and how it can be leveraged to retrieve all activities that occurred across a tenancy. After each retrieval, the results can be indexed into an SIEM tool for further processing.

6       {C}References

OCI Audit Introduction - https://cloud.oracle.com/en_US/governance/audit/features

OCI Audit Overview - https://docs.us-phoenix-1.oraclecloud.com/Content/Audit/Concepts/auditoverview.htm

OCI CLI - https://docs.us-phoenix-1.oraclecloud.com/Content/API/SDKDocs/cliconfigure.htm

OCI Python SDK - https://oracle-cloud-infrastructure-python-sdk.readthedocs.io/en/latest/api/index.html

ListEvents API - https://docs.us-phoenix-1.oraclecloud.com/api/#/en/audit/20160918/AuditEvent/ListEvents

Introduction to OCI IAM - https://cloud.oracle.com/en_US/governance/identity/features

IAM concepts - https://docs.us-phoenix-1.oraclecloud.com/Content/Identity/Concepts/overview.htm

Github - <link to python script>

Cloud Evengelist EMEA

Saurabh belongs to the generation of enthusiastic Millennials. He completed a Master's in Computer Science from The University of Texas, Dallas.

Currently, Saurabh is a Cloud Evangelist as part of the OCI Outbound Product Management team for EMEA. Saurabh spent four years at AWS; prior to that he also worked at HP and LinkedIn.

More about Saurabh Bangad

Share