Schedule Google Compute Engine VM Instance to Start and Stop

October 28, 2020

Flow

  • Cloud Scheduler will send start/stop event at specific time via Pub/Sub
  • The Pub/Sub event will trigger a Cloud Function, where it will start/stop the vm instances via @google-cloud/compute library.

NOTE: Technically Cloud Scheduler could call Cloud Function directly without Pub/Sub, but anyone who figure out the Cloud Function url could start/stop the vm instances (or you need to take extra steps to secure the url)

Create Compute Engine VM instance

Goto https://console.cloud.google.com/compute/instances to create a vm instance for testing.

  • Click Create instance
  • Name: test-schedule
  • Region: us-west1
  • Machine configuration: e2-micro
  • Zone: us-west1-b
  • Expand the Management, security, disks, networking, sole tenancy section.
    • Under Management, click Add label. Enter schedule for Key and test for Value.
  • Click Create

Create Cloud Function

Goto https://console.cloud.google.com/functions to create cloud functions to start/stop the vm instances

Create Start Function

  • Click Create Function
  • Function name: startInstancePubSub
  • Region: default
  • Trigger type: Cloud Pub/Sub
  • Select a Cloud Pub/Sub topic: select Create a topic....
    • Topic ID: start-instance-event
    • Click Create Topic
  • Click Save
  • Click Next
  • Runtime: Node.js 10
  • Entry point: startInstancePubSub
  • Select index.js and paste the following code.
const Compute = require('@google-cloud/compute');
const compute = new Compute();

/**
 * Starts Compute Engine instances.
 *
 * Expects a PubSub message with JSON-formatted event data containing the
 * following attributes:
 *  zone - the GCP zone the instances are located in.
 *  label - the label of instances to start.
 *
 * @param {!object} event Cloud Function PubSub message event.
 * @param {!object} callback Cloud Function PubSub callback indicating
 *  completion.
 */
exports.startInstancePubSub = async (event, context, callback) => {
  try {
    const payload = _validatePayload(
      JSON.parse(Buffer.from(event.data, 'base64').toString())
    );
    // get vm instanaces by labels/metadata rather than name
    // more convinient to change vm metadata rather changing cloud functions code
    const options = {filter: `labels.${payload.label}`};
    const [vms] = await compute.getVMs(options);
    let vmCount = 0;
    await Promise.all(
      vms.map(async (instance) => {
        console.log(`instance=${instance.name}, zone=${instance.zone.id}`)
        if (payload.zone === instance.zone.id) {
          const [operation] = await compute
            .zone(payload.zone)
            .vm(instance.name)
            .start();
          vmCount += 1
          // Operation pending
          return operation.promise();
        }
      })
    );

    // Operation complete. Instance successfully started.
    const message = `Successfully started ${vmCount} instance(s)`;
    console.log(message);
    callback(null, message);
  } catch (err) {
    console.log(err);
    callback(err);
  }
};

/**
 * Validates that a request payload contains the expected fields.
 *
 * @param {!object} payload the request payload to validate.
 * @return {!object} the payload object.
 */
const _validatePayload = (payload) => {
  if (!payload.zone) {
    throw new Error(`Attribute 'zone' missing from payload`);
  } else if (!payload.label) {
    throw new Error(`Attribute 'label' missing from payload`);
  }
  return payload;
};
  • Select package.json and paste the following.
{
  "name": "cloud-functions-schedule-instance",
  "version": "0.0.1",
  "private": true,
  "engines": {
    "node": ">=10.0.0"
  },
  "dependencies": {
    "@google-cloud/pubsub": "^0.18.0",
    "@google-cloud/compute": "^2.0.0"
  }
}
  • Click Deploy

Create Stop Function

  • Click Create Function
  • Function name: stopInstancePubSub
  • Region: default
  • Trigger type: Cloud Pub/Sub
  • Select a Cloud Pub/Sub topic: select Create a topic....
    • Topic ID: stop-instance-event
    • Click Create Topic
  • Click Save
  • Click Next
  • Runtime: Node.js 10
  • Entry point: stopInstancePubSub
  • Select index.js and paste the following code.
const Compute = require('@google-cloud/compute');
const compute = new Compute();

/**
 * Stops Compute Engine instances.
 *
 * Expects a PubSub message with JSON-formatted event data containing the
 * following attributes:
 *  zone - the GCP zone the instances are located in.
 *  label - the label of instances to stop.
 *
 * @param {!object} event Cloud Function PubSub message event.
 * @param {!object} callback Cloud Function PubSub callback indicating completion.
 */
exports.stopInstancePubSub = async (event, context, callback) => {
  try {
    const payload = _validatePayload(
      JSON.parse(Buffer.from(event.data, 'base64').toString())
    );
    // get vm instanaces by labels/metadata rather than name
    // more convinient to change vm metadata rather changing cloud functions code
    const options = {filter: `labels.${payload.label}`};
    const [vms] = await compute.getVMs(options);
    let vmCount = 0;
    await Promise.all(
      vms.map(async (instance) => {
        console.log(`instance=${instance.name}, zone=${instance.zone.id}`)
        if (payload.zone === instance.zone.id) {
          const [operation] = await compute
            .zone(payload.zone)
            .vm(instance.name)
            .stop();
          vmCount += 1
          // Operation pending
          return operation.promise();
        } else {
          return Promise.resolve();
        }
      })
    );

    // Operation complete. Instance successfully stopped.
    const message = `Successfully stopped ${vmCount} instance(s)`;
    console.log(message);
    callback(null, message);
  } catch (err) {
    console.log(err);
    callback(err);
  }
};

/**
 * Validates that a request payload contains the expected fields.
 *
 * @param {!object} payload the request payload to validate.
 * @return {!object} the payload object.
 */
const _validatePayload = (payload) => {
  if (!payload.zone) {
    throw new Error(`Attribute 'zone' missing from payload`);
  } else if (!payload.label) {
    throw new Error(`Attribute 'label' missing from payload`);
  }
  return payload;
};
  • Select package.json and paste the following.
{
  "name": "cloud-functions-schedule-instance",
  "version": "0.0.1",
  "private": true,
  "engines": {
    "node": ">=10.0.0"
  },
  "dependencies": {
    "@google-cloud/pubsub": "^0.18.0",
    "@google-cloud/compute": "^2.0.0"
  }
}

Test the function

{"data":"eyJ6b25lIjoidXMtd2VzdDEtYiIsICJsYWJlbCI6InNjaGVkdWxlPXRlc3QifQ=="}

NOTE: The above data is base64-encoded string for {"zone":"us-west1-b", "label":"schedule=test"}. If you use a different zone (for compute engine) or different labels/metadata, use an online base64 encoding tool to replace the string.

  • Click Test the function.
  • You should see Successfully stopped 1 instance(s) in the output.
  • Goto https://console.cloud.google.com/compute/instances to check if the instances are stopped.
  • You might want to verify if the instance have the right labels. Check the instance, click Show Info Panel and Click Labels.

Setup Cloud Scheduler

Goto https://console.cloud.google.com/cloudscheduler

Create Start Job

  • Click Create Job.
  • Region: default
  • Click Next
  • Name: startup-instances
  • Frequency: 0 9 * * 1-5 (9am, Mon-Fri)
  • Timezone: up to you
  • Target: Pub/Sub
  • Topic: start-instance-event
  • Payload: {"zone":"us-west1-b","label":"schedule=test"}
  • Click Create

Create Stop Job

  • Click Create Job.
  • Region: default
  • Click Next
  • Name: shutdown-instances
  • Frequency: 0 17 * * 1-5 (5pm, Mon-Fri)
  • Timezone: up to you
  • Target: Pub/Sub
  • Topic: stop-instance-event
  • Payload: {"zone":"us-west1-b","label":"schedule=test"}
  • Click Create

Test Scheduler

Click Run now to test startup-instances or shutdown-instances.

VM Startup Script

You might want to configure Auto Start Python Script on Boot

References:

This work is licensed under a
Creative Commons Attribution-NonCommercial 4.0 International License.