All Products
Search
Document Center

SAP HANA Scale-out Deployment Guide

Last Updated: May 16, 2020

SAP HANA Scale-out Deployment Guide

Release history

Version Revision date Changes Release date
1.0 2018-05-14
1.1 2018-05-31 1. Supported the installation of ECS Metrics Collector.
2. Adjusted the layout of some pages.
2018-06-01
1.2 2019-07-04 Optimized Alibaba Cloud Network Attached Storage (NAS) parameters. 2019-07-04

Overview

This topic describes SAP HANA scale-out deployment on Alibaba Cloud. The deployment architecture is based on non-shared storage with one master node and two worker nodes. By using the automatic recovery feature of Elastic Compute Service (ECS), this architecture supports automatic recovery of failed nodes. Therefore, you do not need secondary nodes for SAP HANA instances.

Architecture

createvpc-1

Resource planning

Network

Network Location CIDR block VSwitch VPC
Business network China (Beijing) Zone F 192.168.10.0/24 sap_business SAP_Network

SAP and hosts

SAP HANA (SID: BWS)

Hostname IP address Type Instance ID System ID Description
hana01 192.168.10.4 Master 00 BWS N/A
hana02 192.168.10.5 Worker 00 BWS N/A
hana03 192.168.10.6 Worker 00 BWS N/A

Users and groups

Ensure that all nodes in the SAP HANA cluster use the same user ID and user group ID.
User ID: Set the [sid]adm user ID to 2000.
Group ID: Set the sapsys group ID to 2000.

Swap space

The swap space is required for installing SAP applications and SAP HANA. We recommend that you create an SSD to provide the swap space when you create an ECS instance. For more information about swap space, see
SAP Note 1597355 - Swap-space recommendation for Linux. An SAP service account is required to access this page.

Physical memory (RAM) Recommended swap space
< 32 GB 2 times the amount of RAM
32 to 63 GB 64 GB
64 to 127 GB 96 GB
128 to 255 GB 128 GB
256 to 511 GB 160 GB
512 to 1,023 GB 192 GB
1,024 to 2,047 GB 224 GB
2,048 to 4,095 GB 256 GB
4,096 to 8,191 GB 288 GB
> 8,192 GB 320 GB

File systems

We recommend that you use Autofs to mount global file systems. For more information about file system sizing for SAP HANA, seeSAP HANA TDI-Storage Requirements.In this example, the size of the ECS physical memory (RAM) is 128 GB. The following file system sizing is only for reference:

/hana/data

Sizedata = 3 x RAM

/hana/log

[systems ≤ 512 GB] Sizeredolog = 1/2 x RAM
[systems > 512 GB] Sizeredolog(min) = 512 GB

/hana/shared

Sizeinstallation(scale-out) = 1 x RAM_of_worker per 4 worker nodes

/hana/backup

Sizebackups ≥ Sizedata + Sizeredolog

File system Type Logical volume Volume group Size Disk type
/hana/data XFS datalv hanavg 384 GB SSD
/hana/log XFS loglv hanavg 64 GB SSD
/hana/shared NAS N/A N/A 128 GB SSD
/hana/backup NAS N/A N/A 448 GB Ultra disk
Swap Swap N/A N/A 96 GB SSD

Preparations

Alibaba Cloud account

If you do not have an Alibaba Cloud account, register one on the Alibaba Cloud official website or in the Alibaba Cloud app. You need to use a mobile number to register the account and complete real-name verification for the account. Then, you can log on to the Alibaba Cloud app with this account to manage and monitor your cloud resources, perform authentication, and ask questions and acquire knowledge in Yunqi Community.For more information about how to register and recharge an account, see Account registration and real-name verification (Alibaba Cloud app).

VPC

Virtual Private Cloud (VPC) is an isolated network environment built on Alibaba Cloud. VPCs are logically isolated from one another. A VPC is a private network dedicated to you on Alibaba Cloud. You can configure the IP address range, routing table, and gateway to customize your VPC. For more information, see Virtual Private Cloud.

The following figure shows how to create a VPC as planned.

vpc


The following figure shows how to create a VSwitch and specify a CIDR block for the business network as planned.

vswitch

ECS instance

ECS is a basic cloud computing service provided by Alibaba Cloud.You can log on to the ECS console or Alibaba Cloud app to configure your ECS instances. For more information about SAP NetWeaver and SAP HANA on Alibaba Cloud, see Note 1380654 - SAP support in cloud environments.
1. Create an ECS instance.
Create an ECS instance in the ECS console. Specify the billing method and zone.
Select an existing VPC and an existing security group for the ECS instance. For more information about the security group, see Security group FAQ.
The following table describes the security group rules for this example.

Protocol type Port range Scenario
ICMP -1/-1 Allows ICMP requests.
SSH 22/22 Allows remote Linux SSH access.
Telnet 23/23 Allows requests from default Telnet ports.
HTTPS 443/443 Allows access from HTTPS websites.
SAP HANA 30015-39915 Allows access from SAP HANA.
VNC 5801-5904 Allows access from VNC Viewer.

ecs1
Specify the number and capacity of disks to be created as planned. For more information about disks, see Block storage performance.
We recommend that you use an ultra disk as the system disk and SSDs as data disks, and create an SSD to provide the swap space.

ecs2

  1. Specify the image.
    Select the SUSE Linux Enterprise Server for SAP Applications 12 SP2 image from the image marketplace.
    suseimage

(Optional) Specify the RAM users as required. For more information about RAM, see RAM introduction.
Check all configuration items and ensure that they are correct. Then, create the ECS instance. According to the planning of this deployment, you need to create three ECS instances. After the three ECS instances are created, update hostnames or private IP addresses as planned.

ECS Metrics Collector

ECS Metrics Collector is a monitor agent that the SAP system uses on Alibaba Cloud to collect required information about virtual machine configuration and underlying physical resource usage.

When the SAP system runs in an ECS instance, the SAP Host Agent uses the metadata service and APIs to obtain the information required for monitoring the SAP system, including the information about the operating system, network, storage, and SAP architecture. Then, the SAP Host Agent provides the information for SAP applications to analyze events and system performance.

You need to install ECS Metrics Collector for SAP for each ECS instance in which the SAP system is running, either for SAP applications or SAP HANA.

For more information, see ECS Metrics Collector for SAP deployment guide.

NAS

Alibaba Cloud NAS is a file storage service for compute nodes, such as ECS instances, Elastic High Performance Computing (E-HPC) clusters, and Container Service clusters. NAS complies with standard file access protocols. Without modifying existing applications, you can have a distributed file system that features unlimited capacity and performance scaling, a single namespace, shared access, high reliability, and high availability. For more information about NAS, see Network Attached Storage.
1. Create an NAS file system.
Select a region and a storage type. This deployment uses the NAS Performance storage type. For more information about NAS performance, see Storage types.
In the ECS console, choose Storage & Snapshots > NAS. On the page that appears, specify the region, storage type, capacity, and zone. Select Create new FS and bind naspackage. The following figure shows the configuration for this example.
nas1
After you paid for the NAS storage package, go to the ECS console and refresh the console. The created file system is displayed on the page.
nas2

2. Record the mount addresses of NAS file systems.
Create two NAS file systems for /hana/shared and /hana/backup as planned.
nas3

The following table lists the two NAS file systems in this example.

NAS file system Mount address
/hana/shared 194874b95d-qgi86.cn-beijing.nas.aliyuncs.com
/hana/backup 1207e4a953-gus54.cn-beijing.nas.aliyuncs.com

ECS automatic recovery

The automatic recovery feature is used to improve the availability of ECS instances. This feature reduces influences of physical failures on services.
In the event that the underlying hardware on which your ECS instances are hosted works abnormally or crashes unexpectedly, Alibaba Cloud sends an SMS message to you if the breakdown is confirmed as irreversible.
After the migration is complete, Alibaba Cloud will send a message to indicate that migration is completed. The ECS instance metadata, such as the instance ID, private network IP address, and public network IP address, will remain unchanged.
For more information, see Automatic recovery of instances.

SAP HANA installation

Modify hostnames

Modify hostnames for all nodes of SAP applications and SAP HANA.
Add the following information to the hosts file in the /etc directory as planned:

  1. 192.168.10.4 hana01
  2. 192.168.10.5 hana02
  3. 192.168.10.6 hana03

Create file systems

Create NAS file systems for /hana/shared and /hana/backup. Create local XFS file systems for /hana/data and /hana/log.

1. Create the /hana/data and /hana/log file systems.
(1) Check disks.
(2) Create a physical volume.
(3) Create the hanavg volume group.
(4) Create the datalv and loglv logical volumes.
(5) Create the /hana/data and /hana/log file systems.
(6) Add mount points for the file systems. Enable the file systems to be mounted upon system startup.

  1. fdisk -l
  2. pvcreate /dev/vdb /dev/vdc /dev/vdd
  3. vgcreate hanavg /dev/vdb /dev/vdc /dev/vdd
  4. lvcreate -L 384G -n datalv -i 3 -I 64 hanavg
  5. lvcreate -L 64G -n loglv -i 3 -I 64 hanavg
  6. mkfs.xfs /dev/hanavg/datalv
  7. mkfs.xfs /dev/hanavg/loglv
  8. mkdir -p /hana/data
  9. mkdir -p /hana/log

Run the vi /etc/fstab command to edit the fstab file as follows:

  1. /dev/mapper/hanavg-datalv /hana/data xfs nobarrier,noatime,nodiratime,logbsize=256k 0 0
  2. /dev/mapper/hanavg-loglv /hana/log xfs nobarrier,noatime,nodiratime,logbsize=256k 0 0


Run the mount -a command to mount all file systems.
2. Create the swap space.
(1) Check the disk used to provide the swap space.
Run the lsblk command to obtain disk information. In this example, /dev/vdd provides the swap space.
(2) Configure the swap space as follows:

  1. mkswap /dev/vde
  2. swapon /dev/vde
  3. swapon -s # Checks the size of the swap space.#

3. Mount global file systems.
We recommend that you use Autofs to mount the /hana/shared and /hana/backup file systems. In this way, you do not need to create directories as mount points.
To configure Autofs, follow these steps:
(1) Run the following command to edit the auto.master file:
#vim /etc/auto.master
Add /- /etc/auto.nfs.

(2) Create and edit the auto.nfs file in the /etc directory as follows:

  1. /hana/shared -rw,hard,intr,noresvport,timeo=60,retrans=2 194874b95d-qgi86.cn-beijing.nas.aliyuncs.com:/
  2. /hana/backup -rw,hard,intr,noresvport,timeo=60,retrans=2 1207e4a953-gus54.cn-beijing.nas.aliyuncs.com:/

(3) Run the following command to start Autofs:
#systemctl start autofs
(4) Run the following command to enable Autofs to mount file systems upon system startup:
#systemctl enable autofs
You can run the cd command to access the two file systems to check whether they are mounted.

Prepare the operating system and installation package

You need to configure all nodes of the cluster. The following procedure shows how to configure a node.

1. Install the package required by HA configuration and optimization.
Install the saptune package.
#zypper install -y saptune
Run the following command to check whether this package is installed:

  1. zypper se saptune

saptune

2. Check the Network Time Protocol (NTP) service.
#ntpq -p
By default, the NTP service is enabled for Alibaba ECS instances. If the time zone of your ECS instances is not Asia/Shanghai, change the time zone and configure the NTP service. Ensure that all ECS instances enable the NTP service and use the same time zone. For more information, see Time setting: Synchronize NTP servers and change time zone for Linux instances.

3. Install saptune.
As an upgraded version of the sapconf tool, saptune is available to SUSE Linux Enterprise Server 12 SP2 and later versions. You can use saptune to tune parameters for operating systems and databases. This ensures better performance for SAP NetWeaver or SAP HANA.
The syntax is as follows:
SAP note

  1. Tune system according to SAP and SUSE notes:
  2. saptune note [ list | verify ]
  3. saptune note [ apply | simulate | verify | customise | revert ] NoteID

SAP solution

  1. Tune system for all notes applicable to your SAP solution:
  2. saptune solution [ list | verify ]
  3. saptune solution [ apply | simulate | verify | revert ] SolutionName

For this deployment, SAP HANA is installed. The following figure shows the installation results of saptune.
installsummary

Run the following command to start a daemon:
#saptune daemon start

For more information about saptune, see Prepare your Linux for your SAP solution with saptune or the official documentation for SUSE Linux Enterprise Server.

Install SAP HANA

  1. Install HANA on the primary node.
  1. hana01:/sapcd/HANA2.0_SPS02_THIN/HDB_SERVER_LINUX_X86_64 # ./hdblcm
  2. SAP HANA Lifecycle Management - SAP HANA Database 2.00.020.00.1500920972
  3. ************************************************************************
  4. Scanning software locations...
  5. Detected components:
  6. SAP HANA Database (2.00.020.00.1500920972) in /sapcd/HANA2.0_SPS02_THIN/HDB_SERVER_LINUX_X86_64/server
  7. SAP HANA AFL (incl.PAL,BFL,OFL,HIE) (2.00.020.0000.1500932993) in /sapcd/HANA2.0_SPS02_THIN/HDB_AFL_LINUX_X86_64/packages
  8. SAP HANA EML AFL (2.00.020.0000.1500932993) in /sapcd/HANA2.0_SPS02_THIN/HDB_EML_AFL_10_LINUX_X86_64/packages
  9. SAP HANA Database Client (2.2.23.1499440855) in /sapcd/HANA2.0_SPS02_THIN/HDB_CLIENT_LINUX_X86_64/client
  10. Choose an action
  11. Index | Action | Description
  12. -----------------------------------------------
  13. 1 | install | Install new system
  14. 2 | extract_components | Extract components
  15. 3 | Exit (do nothing) |
  16. Enter selected action index [3]: 1
  17. SAP HANA Database version '2.00.020.00.1500920972' will be installed.
  18. Select additional components for installation:
  19. Index | Components | Description
  20. -------------------------------------------------------------------------------------------------
  21. 1 | all | All components
  22. 2 | server | No additional components
  23. 3 | client | Install SAP HANA Database Client version 2.2.23.1499440855
  24. 4 | afl | Install SAP HANA AFL (incl.PAL,BFL,OFL,HIE) version 2.00.020.0000.1500932993
  25. 5 | eml | Install SAP HANA EML AFL version 2.00.020.0000.1500932993
  26. Enter comma-separated list of the selected indices [3]: 2,3
  27. Enter Installation Path [/hana/shared]: /hana/shared
  28. Enter Local Host Name [hana01]:
  29. Do you want to add hosts to the system? (y/n) [n]:
  30. Enter SAP HANA System ID: BWS
  31. Enter Instance Number [00]: 00
  32. Enter Local Host Worker Group [default]:
  33. Index | System Usage | Description
  34. -------------------------------------------------------------------------------
  35. 1 | production | System is used in a production environment
  36. 2 | test | System is used for testing, not production
  37. 3 | development | System is used for development, not production
  38. 4 | custom | System usage is neither production, test nor development
  39. Select System Usage / Enter Index [4]: 4
  40. Enter Location of Data Volumes [/hana/data/BWS]:
  41. Enter Location of Log Volumes [/hana/log/BWS]:
  42. Restrict maximum memory allocation? [n]:
  43. Enter Certificate Host Name For Host 'hana01' [hana01]:
  44. Enter SAP Host Agent User (sapadm) Password:
  45. Confirm SAP Host Agent User (sapadm) Password:
  46. Enter System Administrator (bwsadm) Password:
  47. Confirm System Administrator (bwsadm) Password:
  48. Enter System Administrator Home Directory [/usr/sap/BWS/home]:
  49. Enter System Administrator Login Shell [/bin/sh]:
  50. Enter System Administrator User ID [1001]: 2000
  51. Enter ID of User Group (sapsys) [79]: 2000
  52. Enter System Database User (SYSTEM) Password:
  53. Confirm System Database User (SYSTEM) Password:
  54. Restart system after machine reboot? [n]:
  55. Summary before execution:
  56. =========================
  57. SAP HANA Database System Installation
  58. Installation Parameters
  59. Remote Execution: ssh
  60. Database Isolation: low
  61. Installation Path: /hana/shared
  62. Local Host Name: hana01
  63. SAP HANA System ID: BWS
  64. Instance Number: 00
  65. Local Host Worker Group: default
  66. System Usage: custom
  67. Location of Data Volumes: /hana/data/BWS
  68. Location of Log Volumes: /hana/log/BWS
  69. Certificate Host Names: hana01 -> hana01
  70. System Administrator Home Directory: /usr/sap/BWS/home
  71. System Administrator Login Shell: /bin/sh
  72. System Administrator User ID: 2000
  73. ID of User Group (sapsys): 2000
  74. SAP HANA Database Client Installation Path: /hana/shared/BWS/hdbclient
  75. Software Components
  76. SAP HANA Database
  77. Install version 2.00.020.00.1500920972
  78. Location: /sapcd/HANA2.0_SPS02_THIN/HDB_SERVER_LINUX_X86_64/server
  79. SAP HANA AFL (incl.PAL,BFL,OFL,HIE)
  80. Do not install
  81. SAP HANA EML AFL
  82. Do not install
  83. SAP HANA Database Client
  84. Install version 2.2.23.1499440855
  85. Location: /sapcd/HANA2.0_SPS02_THIN/HDB_CLIENT_LINUX_X86_64/client
  86. Do you want to continue? (y/n):

Modify configurations to meet requirements of the non-shared storage architecture.
vim /hana/shared/[SID]/global/hdb/custom/config/global.ini

Add the following configuration:

  1. basepath_shared=no

hanaprofile

Change the internal communication mode. Run the./hdblcm —action=configure_internal_network commandas a root user to specify the CIDR block for communication. In this example, the communication CIDR block is 192.168.10/24.
hanaprofile

  1. Install HANA on worker nodes. The following use one worker node as example. The installation of the other worker node is similar.
    Create paths for instances.
    mkdir -p /hana/data/[HANA SID]
    mkdir -p /hana/log/[HANA SID]
    Modify directory permissions.
    chmod -R 777 /hana/data/[HANA SID]
    chmod -R 777 /hana/log/[HANA SID]

Install SAP HANA on the worker node.
Run the cd command as a root user to access the installation path.cd /hana/shared/[HANA SID]/hdblcm

  1. #cd /hana/shared/BWS/hdblcm
  2. #./hdblcm
  3. SAP HANA Lifecycle Management - SAP HANA Database 2.2.18.0
  4. **********************************************************
  5. Choose an action
  6. Index | Action | Description
  7. -----------------------------------------------------------------------------
  8. 1 | add_hosts | Add Hosts to the SAP HANA Database System
  9. 2 | register_rename_system | Register and Rename SAP HANA Database System
  10. 3 | exit | Exit (do nothing)
  11. Enter selected action index [3]: 1
  12. Enter comma-separated host names to add: hana02
  13. Enter SAP Host Agent User (sapadm) Password:
  14. Select roles for host 'hana02':
  15. Index | Host Role | Description
  16. -------------------------------------------------------------------
  17. 1 | worker | Database Worker
  18. 2 | standby | Database Standby
  19. 3 | extended_storage_worker | Dynamic Tiering Worker
  20. 4 | extended_storage_standby | Dynamic Tiering Standby
  21. 5 | streaming | Streaming Analytics
  22. 6 | rdsync | Remote Data Sync
  23. 7 | ets_worker | Accelerator for SAP ASE Worker
  24. 8 | ets_standby | Accelerator for SAP ASE Standby
  25. 9 | xs_worker | XS Advanced Runtime Worker
  26. 10 | xs_standby | XS Advanced Runtime Standby
  27. Enter comma-separated list of selected indices [1]: 1
  28. Enter Host Failover Group for host 'hana02' [default]:
  29. Enter Storage Partition Number for host 'hana02' [<<assign automatically>>]:
  30. Enter Worker Group for host 'hana02' [default]:
  31. Enter System Administrator (bwsadm) Password:
  32. Enter Certificate Host Name For Host 'hana02' [hana02]:
  33. Summary before execution:
  34. =========================
  35. Add Hosts to SAP HANA Database System
  36. Add Hosts Parameters
  37. Skip all SAP Host Agent calls: No
  38. Remote Execution: ssh
  39. Enable the installation or upgrade of the SAP Host Agent: Yes
  40. Auto Initialize Services: Yes
  41. Do not start added hosts and do not start SAP HANA Database System: No
  42. Certificate Host Names: hana02 -> hana02
  43. Do not Modify '/etc/sudoers' File: No
  44. Additional Hosts
  45. hana02
  46. Role: Database Worker (worker)
  47. High-Availability Group: default
  48. Worker Group: default
  49. Storage Partition: <<assign automatically>>
  50. Do you want to continue? (y/n): y
  1. Modify parameters to enable SAP HANA to start upon system startup.
    Log on to the SAP HANA hosts as a root user and modify the Autostart parameter.
    vim /hana/shared/[SID]/profile/[SID]_HDB[instance number]_[host name]
    Change the value of the Autostart parameter from 0 to 1. Save the settings and restart SAP HANA instances.
    In this example, you need to modify the parameter for three instances.
    autostart

Repeat the preceding steps to install SAP HANA on the other worker node according to the planning for this example.

Verify the installation

  1. Log out the root account and log on to the ECS instances as the administrator.su -[sid]adm
  2. Verify that the cluster is deployed.
    cd /hana/shared/[SID]/exe/linuxx86_64/hdb/python_support/
    watch python landscapeHostConfiguration.py
    landscapehost