## Autoscaling with Heat

updated: 2020-10-17 18:54

# Autoscaling with Heat

## Goal

There are Senlin resource types in Heat which make deployment of a full-featured auto-scaling solution easily attainable. This document is to provide a tutorial for users who want to use heat to create a senlin cluster.

It is often required by real deployment practices to make the cluster load-balanced and auto-scaled. We also want the scaling action triggered based on business data instead of infrastructure metrics. When existing cluster is not enough to afford the throughput/workload, the cluster will be scaled-out; when low throughput or workload, the cluster will be scaled-in.

Moreover, custom is easy to do when auto-scaling. Receivers can be created to generate webhooks from scale_out and scale_in actions. Moreover, placement_zone.yaml and placement_region.yaml can be attached to cluster and guide which zone/region to place new nodes when scale_out; deletion_policy can be attached to the cluster and guide the choice of candidates to delete when scale_in.

## Sample template

There have a sample template in heat-template project under directory of senlin for creation of Senlin elastic load-balanced cluster by Heat. Here we choose some important parts of the sample to explain one by one.

The resource below defines a security_group for connection to created load-balanced cluster:

    security_group:
      type: OS::Neutron::SecurityGroup
      properties:
        rules:
          - protocol: icmp
          - protocol: tcp
            port_range_min: 22
            port_range_max: 22
          - protocol: tcp
            port_range_min: 80
            port_range_max: 80

The resource below defines the profile used to create the targeted cluster:

    profile:
      type: OS::Senlin::Profile
      properties:
        type: os.nova.server-1.0
        properties:
          flavor: {get_param: flavor}
          image: {get_param: image}
          key_name: {get_param: key_name}
          networks:
            - network: {get_param: network}
          security_groups:
            - {get_resource: security_group}

The resource below defines to create a Senlin cluster with two nodes at least:

    cluster:
      type: OS::Senlin::Cluster
      properties:
        desired_capacity: 2
        min_size: 2
        profile: {get_resource: profile}

The two resources below define scale_in_policy and scale_out_policy attached to the created cluster. Where, the property of event is used to define the objective action the policy works. When type of the property of adjustment is set as CHANGE_IN_CAPACITY, the cluster will increase the number of nodes when scale_out or decrease the number of nodes when scale_in:

    scale_in_policy:
      type: OS::Senlin::Policy
      properties:
        type: senlin.policy.scaling-1.0
        bindings:
          - cluster: {get_resource: cluster}
        properties:
          event: CLUSTER_SCALE_IN
          adjustment:
            type: CHANGE_IN_CAPACITY
            number: 1

    scale_out_policy:
      type: OS::Senlin::Policy
      properties:
        type: senlin.policy.scaling-1.0
        bindings:
          - cluster: {get_resource: cluster}
        properties:
          event: CLUSTER_SCALE_OUT
          adjustment:
            type: CHANGE_IN_CAPACITY
            number: 1

The resource below defines a lb_policy to be attached to the target cluster. Once the policy is attached to the cluster, Senlin will automatically create loadbalancer, pool, and health_monitor by invoking neutron LBaas V2 APIs for load-balancing purpose:

    lb_policy:
      type: OS::Senlin::Policy
      properties:
        type: senlin.policy.loadbalance-1.0
        bindings:
          - cluster: {get_resource: cluster}
        properties:
          pool:
            protocol: HTTP
            protocol_port: 80
            subnet: {get_param: pool_subnet}
            lb_method: ROUND_ROBIN
          vip:
            subnet: {get_param: vip_subnet}
            protocol: HTTP
            protocol_port: 80
          health_monitor:
            type: HTTP
            delay: 10
            timeout: 5
            max_retries: 4

The two resources below define the receivers to be triggered when a certain alarm or event occurs:

    receiver_scale_out:
      type: OS::Senlin::Receiver
      properties:
        cluster: {get_resource: cluster}
        action: CLUSTER_SCALE_OUT
        type: webhook

    receiver_scale_in:
      type: OS::Senlin::Receiver
      properties:
        cluster: {get_resource: cluster}
        action: CLUSTER_SCALE_IN
        type: webhook

The resource below define the policy for selecting candidate nodes for deletion when the cluster is to be shrank:

    deletion_policy:
      type: OS::Senlin::Policy
      properties:
        type: senlin.policy.deletion-1.0
        bindings:
          - cluster: {get_resource: cluster}
        properties:
          criteria: YOUNGEST_FIRST
          destroy_after_deletion: True
          grace_period: 20
          reduce_desired_capacity: False

The two resources below define the alarms to trigger the above two receivers respectively. We use the average rate of incoming bytes at LoadBalancer as the metrics to trigger the scaling operations:

    scale_in_alarm:
      type: OS::Ceilometer::Alarm
      properties:
        description: trigger when bandwidth overflow
        meter_name: network.services.lb.incoming.bytes.rate
        statistic: avg
        period: 180
        evaluation_periods: 1
        threshold: 12000
        repeat_actions: True
        alarm_actions:
          - {get_attr: [receiver_scale_in, channel, alarm_url]}
        comparison_operator: le
        query:
          metadata.user_metadata.cluster_id: {get_resource: cluster}

    scale_out_alarm:
      type: OS::Ceilometer::Alarm
      properties:
        description: trigger when bandwidth insufficient
        meter_name: network.services.lb.incoming.bytes.rate
        statistic: avg
        period: 60
        evaluation_periods: 1
        threshold: 28000
        repeat_actions: True
        alarm_actions:
          - {get_attr: [receiver_scale_out, channel, alarm_url]}
        comparison_operator: ge
        query:
          metadata.user_metadata.cluster_id: {get_resource: cluster}

## Deployment Steps

Before the deployment, please ensure that neutron LBaas v2 and ceilometer/Aodh has been installed and configured in your environment.

Step one is to generate key-pair using the followed command:

    $ openstack keypair create heat_key

Step two is to create a heat template as by downloading the template file from heat template.

Step three is to create a heat stack using the followed command:

    $ openstack stack create test -t ./ex_aslb.yaml --parameter "key_name=heat_key"

The steps and samples introduced in this tutorial can also work well together with composition of ceilometer, Aodh, and Gnocchi without any change.

updated: 2020-10-17 18:54

Except where otherwise noted, this document is licensed under Creative Commons Attribution 3.0 License. See all OpenStack Legal Documents.

questions?