X
September 10, 2018

Introducing Updateable Instance Metadata

By: Craig Carl | Director

Share

Starting today customers can update instance metadata on all OCI instances via the OCI API, SDKs and the CLI. Updateable Instance Metadata enables a secure communications channel to compute instances that does not require any externally accessible services. Customers can now more easily build secure compute enclaves for highly sensitive workloads.

Instance metadata and cloud-init are two of the little pieces of magic that make IaaS so compelling. Instance metadata has always been leveraged at initial launch by customers who rely on cloud-init (and for Windows) to configure an instance. That configuration could be a simple `yum update` or it could be used to install an Oracle Management Cloud Agent for advanced monitoring and management. Installing and configuring Chef or Puppet agents, joining an Active Directory domain, much more are all simple to automate with instance metadata.

Here’s what some of the metadata on an instance looks like –

$ curl http://169.254.169.254/opc/v1/instance/
{
  "availabilityDomain" : "Uocm:PHX-AD-2",
  "faultDomain" : "FAULT-DOMAIN-1",
  "compartmentId" : "ocid1.compartment.oc1..aaaaaaaay4bxm4m5k7ii7oqyygolnuyozt5tyb5ufsl2jgcehm4hl4fslrwa",
  "displayName" : "updateable_metadata",
  "id" : "ocid1.instance.oc1.phx.abyhqljrrtcvkpxo33brxsfpykyrfg2n5r6owmyncywppxmt75ou2ap2n2xa",
  "image" : "ocid1.image.oc1.phx.aaaaaaaasez4lk2lucxcm52nslj5nhkvbvjtfies4yopwoy4b3vysg5iwjra",
  "metadata" : {
    "ssh_authorized_keys" : "ssh-rsa AAAAB3NzaC...4cON",
    "user_data" : "V2UncmUgaGlyaW5nLCBnZXQgaW4gdG91Y2ghIGNyYWlnLmNhcmxAb3JhY2xlLmNvbQ=="
  },
  "region" : "phx",
  "canonicalRegionName" : "us-phoenix-1",
  "shape" : "VM.Standard2.1",
  "state" : "Running",
  "timeCreated" : 1536284426464
}

Because instance metadata and cloud-init work so well together we often think about them as being a single thing. They aren’t. Cloud-init is an application that runs the first time an instance is launched, it gets a document from the instance metadata service and processes it per the documentation.

When we decouple instance metadata from cloud-init it becomes obvious that instance metadata is an out-of-band communications channel. Traditionally we interact with compute instances by connecting to services running on the instance that accept inbound connections, SSH and HTTP are two common in-band communications channels. These services introduce security risks; they can contain bugs, they can be misconfigured, they need to be regularly and carefully updated. The same applies to any application on an instance that accepts an inbound connection, they all create risk.

What we need is a secure channel to communicate with a compute resource that doesn’t require any services that listen for external connections. Updateable Instance Metadata gives us this channel. Updateable Instance Metadata eliminates the need for listening services on the compute instance and allows us to leverage the strong OCI IAM permissions and policy features to secure it.

Let’s imagine a dataset that is always encrypted in transit and at rest. Unfortunately, it’s still difficult to leverage encrypted data, it must be decrypted first. Decrypting the data increases the risk of losing control over it. Updateable Instance Metadata enables us to use the data and collect the results from a compute enclave that doesn’t accept any inbound connections. This is a significant security advantage. There are multiple pieces to this solution;

  1. A custom image that includes the analytics software plus a small application that polls the instance metadata. SSH and other services should be disabled, the firewall should be configured to deny all inbound connections. Set the GRUB menu timeout to 0. The custom image should also include a temporary key encryption key (KEK).
  2. A VCN with a private subnet and a Service Gateway. The private subnet isolates the instances from the Internet and the Service Gateway allows outbound access to the OCI Object Store without allowing access elsewhere.
  3. A bucket in the OCI Object Store. This will contain the encrypted dataset(s) as well as the results of the analysis.
  4. A Dynamic Group, matching rule, and IAM policy. These will authorize the instance to GET the data and POST the results to the object store.

Now we can launch any number of instances, we’ll call them workers. When there is a dataset that needs to be processed we will use the OCI API to update the instance metadata on a worker with two key:values; “object”:”<path to object>” and “DEK”:”<data encryption key>”. The DEK should be unique to each individual unit of work. An application on the instance will get the object, decrypt the DEK and then the dataset. Once the analysis is complete the results can be encrypted with the DEK and PUT to the object store.

The OCI API defines two metadata keys for an instance, `metadata` and `extendedMetadata`. The contents of the `metadata` and `extendedMetadata` PUT via the API are merged into the `metadata` key on an instance. Updating the `metadata` key via the API is subject to multiple limitations*, let’s focus on `extendedMetadata`. The maximum size of the combined metadata, including userdata and SSH keys is 31.25 kibibytes.

To update the metadata on our instance with our two new keys we first need to define them. Passing complex JSON on the CLI is difficult so we will source it from a file -

$ cat extended-md.json
{
	"object": "https://objectstorage.us-phoenix-1.oraclecloud.com/p/7GWMRaWucZ-dqIgocR9OVc6dUGiB5QwHX4V-QISkbCI/n/myns/b/money/o/someencypteddata",
	"DEK": ""
}

To apply the update -

$ oci compute instance update --instance-id ocid1.instance.oc1.phx.abyhqljr…n2xa --extended-metadata file://./extended-md.json

When we check the metadata on the instance again we can see our update -

[opc@updateable-metadata ~]$ curl http://169.254.169.254/opc/v1/instance/
{
  "availabilityDomain" : "Uocm:PHX-AD-2",
  "faultDomain" : "FAULT-DOMAIN-1",
  "compartmentId" : "ocid1.compartment.oc1..aaaaaaaay4bxm4m5k7ii7oqyygolnuyozt5tyb5ufsl2jgcehm4hl4fslrwa",
  "displayName" : "updateable_metadata",
  "id" : "ocid1.instance.oc1.phx.abyhqljrrtcvkpxo33brxsfpykyrfg2n5r6owmyncywppxmt75ou2ap2n2xa",
  "image" : "ocid1.image.oc1.phx.aaaaaaaasez4lk2lucxcm52nslj5nhkvbvjtfies4yopwoy4b3vysg5iwjra",
  "metadata" : {
    "DEK" : "",
    "user_data" : "V2UncmUgaGlyaW5nLCBnZXQgaW4gdG91Y2ghIGNyYWlnLmNhcmxAb3JhY2xlLmNvbQ==",
    "object" : "https://objectstorage.us-phoenix-1.oraclecloud.com/p/7GWMRaWucZ-dqIgocR9OVc6dUGiB5QwHX4V-QISkbCI/n/myns/b/money/o/someencypteddata",
    "ssh_authorized_keys" : "ssh-rsa AAAAB3NzaC...4cON"
  },
  "region" : "phx",
  "canonicalRegionName" : "us-phoenix-1",
  "shape" : "VM.Standard2.1",
  "state" : "Running",
  "timeCreated" : 1536284426464
}

Updateable Instance Metadata provides a highly secure, out-of-band communications channel that can be leveraged to build a secure compute enclave for highly sensitive workloads. I’m excited to see what you build with Updateable Instance Metadata, please let me know!

To get started with Updateable Instance Metadata on OCI, visit https://cloud.oracle.com.  Updateable Instance Metadata are available at no additional cost in all public OCI regions and ADs. For more information, see the Oracle Cloud Infrastructure Getting Started guideCompute service overview, and <add link to TC on updating instance metadata>

Craig

* The `metadata` key includes two reserved keys, `user_data` and ` ssh_authorized_keys`. These cannot be updated. If you want to update or add any other keys to the `metadata` key you must include the launch time values for the `user_data` and ` ssh_authorized_keys` in the update. The `metadata` key only supports key:values, no nesting allowed. For all these reasons it’s best to avoid updating the `metadata` key.

Share