Getting Started with SaltStack on AlmaLinux 10: A Complete Beginner's Guide

Introduction

Infrastructure as Code (IaC) has revolutionized how we manage servers and applications. Among the various configuration management tools available, SaltStack stands out for its speed, scalability, and flexibility. In this comprehensive guide, we’ll explore SaltStack from the ground up, with a focus on AlmaLinux 10.

Whether you’re managing a handful of servers or thousands, SaltStack can help you automate configuration, deployment, and orchestration tasks efficiently.

What is SaltStack?

SaltStack (often called “Salt”) is an open-source configuration management and remote execution tool. It uses a master-minion architecture where a central Salt master manages multiple Salt minions (your servers).

Key Features

Fast and Scalable: Built on ZeroMQ, Salt can manage thousands of nodes simultaneously
Event-Driven: Real-time execution and event system for orchestration
Python-Based: Easy to extend with custom modules
Declarative Configuration: Define desired state, Salt ensures it’s maintained
Remote Execution: Run commands across your infrastructure instantly
Flexible: Works with or without agents (masterless mode available)

Why SaltStack?

Use Cases

1. Configuration Management

Ensure all servers have consistent configurations
Manage files, packages, services, and users
Enforce security policies across your infrastructure

2. Application Deployment

Deploy applications to multiple servers simultaneously
Rollback deployments if issues occur
Blue-green and canary deployments

3. Infrastructure Automation

Provision new servers automatically
Configure networking, storage, and security
Integrate with cloud providers (AWS, Azure, GCP)

4. Orchestration

Coordinate complex multi-step deployments
Execute tasks in specific order across multiple servers
React to events in real-time

5. Compliance and Security

Enforce security baselines
Audit configurations
Remediate drift automatically

SaltStack vs Other Tools

Feature	SaltStack	Ansible	Puppet	Chef
Speed	Very Fast	Moderate	Moderate	Moderate
Scalability	Excellent	Good	Good	Good
Agent Required	Yes (can run agentless)	No	Yes	Yes
Learning Curve	Moderate	Easy	Steep	Steep
Language	Python + YAML	YAML	Ruby DSL	Ruby DSL
Real-time Execution	Yes	No	No	No

Core Concepts

1. Master-Minion Architecture

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
┌─────────────────┐
│   Salt Master   │  ← Central management server
│  (Control Node) │
└────────┬────────┘
         │
    ┌────┴────┬────────┬─────────┐
    │         │        │         │
┌───▼───┐ ┌──▼───┐ ┌──▼───┐ ┌──▼───┐
│Minion1│ │Minion2│ │Minion3│ │Minion4│
└───────┘ └──────┘ └──────┘ └──────┘
  Web       DB       Cache     API

Master: Central server that sends commands and configurations to minions
Minions: Servers managed by the master, executing commands and applying configurations

2. States

States are declarative descriptions of how a system should be configured. Written in YAML with Jinja templating.

Example State:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
nginx:
  pkg.installed:
    - name: nginx

nginx-service:
  service.running:
    - name: nginx
    - enable: True
    - require:
        - pkg: nginx

3. Pillar

Pillar stores configuration data specific to minions. Think of it as variables for your infrastructure.

Example Pillar:

1
2
3
4
5
6
7
nginx:
  port: 80
  domain: example.com

database:
  host: db.example.com
  port: 5432

4. Grains

Grains are static information about minions (OS, CPU, memory, network). Collected when minion starts.

1
2
salt 'web-server' grains.items
# Shows: os, osrelease, cpu_model, mem_total, ip_interfaces, etc.

5. Execution Modules

Remote execution functions you can run on minions.

1
2
3
salt '*' cmd.run 'hostname'
salt 'web-*' service.restart nginx
salt 'db-*' pkg.install postgresql

Setting Up Your First Salt Environment

Prerequisites

Two AlmaLinux 10 servers (or VMs)
- 1 for Salt Master (2 CPU, 2GB RAM minimum)
- 1 for Salt Minion (1 CPU, 1GB RAM minimum)
Root or sudo access
Network connectivity between servers

Lab Setup

For this tutorial, we’ll use:

Master: salt-master (192.168.1.10)
Minion: web-server (192.168.1.11)

Step 1: Install Salt Master

SSH to your master server:

`1`	`ssh [email protected]`

Update the system:

1
dnf update -y

Install Salt Master:

1
2
3
4
5
6
7
8
# Add Salt repository
curl -fsSL https://packages.broadcom.com/artifactory/api/security/keypair/SaltProjectKey/public | gpg --dearmor | sudo tee /usr/share/keyrings/salt-archive-keyring.gpg > /dev/null

# Add repository
echo "deb [signed-by=/usr/share/keyrings/salt-archive-keyring.gpg arch=amd64] https://packages.broadcom.com/artifactory/saltproject-deb/ stable main" | sudo tee /etc/apt/sources.list.d/salt.list

# Install (for AlmaLinux, use dnf)
dnf install -y salt-master salt-minion

Start and enable Salt Master:

1
2
systemctl enable --now salt-master
systemctl status salt-master

Configure firewall:

1
2
3
4
# Allow Salt master ports
firewall-cmd --permanent --add-port=4505/tcp  # Salt publish port
firewall-cmd --permanent --add-port=4506/tcp  # Salt return port
firewall-cmd --reload

Verify installation:

1
2
salt --version
# Should show: salt 3006.x

Step 2: Install Salt Minion

SSH to your minion server:

`1`	`ssh [email protected]`

Install Salt Minion:

1
dnf install -y salt-minion

Configure the minion:

1
2
3
4
5
vim /etc/salt/minion

# Add these lines:
master: 192.168.1.10
id: web-server

Start and enable Salt Minion:

1
2
systemctl enable --now salt-minion
systemctl status salt-minion

Step 3: Accept Minion Key

Back on the Salt Master:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
# List unaccepted keys
salt-key -L

# Should show:
# Accepted Keys:
# Denied Keys:
# Unaccepted Keys:
# web-server
# Rejected Keys:

# Accept the minion key
salt-key -a web-server

# Verify
salt-key -L
# Should now show web-server under "Accepted Keys"

Step 4: Test Connection

Test connectivity between master and minion:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
# Test ping
salt 'web-server' test.ping
# Output: web-server: True

# Check minion's OS
salt 'web-server' grains.get os
# Output: web-server: AlmaLinux

# Run a command
salt 'web-server' cmd.run 'hostname'
# Output: web-server: web-server

# Get system info
salt 'web-server' grains.items

Success! Your master and minion are communicating.

Step 5: Create Your First State

Now let’s create a simple state to install and configure nginx.

Create the state directory:

1
mkdir -p /srv/salt/nginx

Create the state file:

1
vim /srv/salt/nginx/init.sls

Add this content:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
# Install nginx
nginx:
  pkg.installed:
    - name: nginx

# Ensure nginx is running
nginx-service:
  service.running:
    - name: nginx
    - enable: True
    - require:
        - pkg: nginx

# Create a simple index page
/usr/share/nginx/html/index.html:
  file.managed:
    - contents: |
        <!DOCTYPE html>
        <html>
        <head>
            <title>Welcome to Salt-Managed Server</title>
            <style>
                body {
                    font-family: Arial, sans-serif;
                    text-align: center;
                    padding: 50px;
                    background: linear-gradient(135deg, #667eea 0%, #764ba2 100%);
                    color: white;
                }
                h1 { font-size: 48px; margin-bottom: 20px; }
                p { font-size: 24px; }
            </style>
        </head>
        <body>
            <h1>🚀 Server Managed by SaltStack</h1>
            <p>Hostname: {{ grains['id'] }}</p>
            <p>OS: {{ grains['os'] }} {{ grains['osrelease'] }}</p>
            <p>Configured automatically with Salt</p>
        </body>
        </html>
    - mode: "0644"
    - user: root
    - group: root
    - template: jinja
    - require:
        - pkg: nginx

# Configure firewall to allow HTTP
http-firewall:
  firewalld.service:
    - name: http
    - enable: True

Create top file (tells Salt which states to apply to which minions):

1
vim /srv/salt/top.sls

Add:

1
2
3
base:
  "web-server":
    - nginx

Step 6: Apply the State

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
# Test what would change (dry run)
salt 'web-server' state.apply test=True

# Apply the state
salt 'web-server' state.apply

# Output shows:
# - nginx package installed
# - nginx service started and enabled
# - index.html file created
# - firewall configured

Step 7: Verify

Open your browser and navigate to:

1
http://192.168.1.11

You should see your custom welcome page showing the server is managed by Salt!

Verify from command line:

1
2
salt 'web-server' cmd.run 'systemctl status nginx'
salt 'web-server' cmd.run 'curl -s localhost | grep SaltStack'

Understanding What Happened

Let’s break down the state file:

1. Package Management

1
2
3
nginx:
  pkg.installed:
    - name: nginx

Uses Salt’s pkg module
Ensures nginx package is installed
Idempotent: runs only if not already installed

2. Service Management

1
2
3
4
5
6
nginx-service:
  service.running:
    - name: nginx
    - enable: True
    - require:
        - pkg: nginx

Ensures nginx service is running
Enables it to start on boot
require creates dependency: service only starts after package installed

3. File Management

1
2
3
4
5
/usr/share/nginx/html/index.html:
  file.managed:
    - contents: |
        ...HTML content...
    - template: jinja

Creates/manages the index.html file
Uses Jinja templating to inject grain values
Sets proper permissions

4. Jinja Templating

1
2
Hostname: {{ grains['id'] }}
OS: {{ grains['os'] }} {{ grains['osrelease'] }}

Dynamic content based on minion’s grains
Can use variables, loops, conditionals

Working with Pillar Data

Let’s make our state more flexible using Pillar.

Create pillar directory:

1
mkdir -p /srv/pillar

Create pillar data:

1
vim /srv/pillar/nginx.sls

Add:

1
2
3
4
nginx:
  port: 80
  server_name: example.com
  root: /usr/share/nginx/html

Create pillar top file:

1
vim /srv/pillar/top.sls

Add:

1
2
3
base:
  "web-server":
    - nginx

Update the state to use pillar:

1
vim /srv/salt/nginx/init.sls

Add nginx configuration:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
# ... previous states ...

# Custom nginx configuration
/etc/nginx/conf.d/default.conf:
  file.managed:
    - source: salt://nginx/files/default.conf.j2
    - template: jinja
    - user: root
    - group: root
    - mode: "0644"
    - require:
        - pkg: nginx

# Reload nginx when config changes
nginx-reload:
  cmd.run:
    - name: nginx -t && systemctl reload nginx
    - onchanges:
        - file: /etc/nginx/conf.d/default.conf

Create template directory and file:

1
2
mkdir -p /srv/salt/nginx/files
vim /srv/salt/nginx/files/default.conf.j2

Add:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
server {
    listen {{ pillar['nginx']['port'] }};
    server_name {{ pillar['nginx']['server_name'] }};
    root {{ pillar['nginx']['root'] }};

    index index.html;

    location / {
        try_files $uri $uri/ =404;
    }
}

Apply the updated state:

1
2
3
4
5
# Refresh pillar
salt 'web-server' saltutil.refresh_pillar

# Apply state
salt 'web-server' state.apply nginx

Remote Execution Examples

Salt’s remote execution is powerful. Here are common tasks:

System Information

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
# Get hostname
salt '*' cmd.run 'hostname'

# Check disk usage
salt '*' disk.usage

# Get network interfaces
salt '*' network.interfaces

# Check memory
salt '*' status.meminfo

Package Management

1
2
3
4
5
6
7
8
# Install a package
salt '*' pkg.install vim

# Update all packages
salt '*' pkg.upgrade

# Remove a package
salt '*' pkg.remove apache2

Service Management

1
2
3
4
5
6
7
8
# Restart a service
salt '*' service.restart nginx

# Check service status
salt '*' service.status nginx

# Enable service on boot
salt '*' service.enable nginx

User Management

1
2
3
4
5
6
7
8
# Create a user
salt '*' user.add john

# Add user to group
salt '*' user.chgroups john groups=wheel append=True

# Remove a user
salt '*' user.delete john

File Operations

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
# Create a directory
salt '*' file.mkdir /opt/myapp

# Copy a file
salt '*' cp.get_file salt://files/config.txt /etc/myapp/config.txt

# Check if file exists
salt '*' file.file_exists /etc/nginx/nginx.conf

# Set file permissions
salt '*' file.set_mode /etc/myapp/config.txt 0600

Targeting Minions

Salt offers flexible targeting options:

Glob Matching (Default)

1
2
3
salt 'web-*' test.ping          # All minions starting with 'web-'
salt '*-prod' test.ping          # All production servers
salt 'web-[1-3]' test.ping       # web-1, web-2, web-3

List Matching

1
salt -L 'web-1,web-2,db-1' test.ping

Grain Matching

1
2
3
salt -G 'os:AlmaLinux' test.ping
salt -G 'osrelease:10' test.ping
salt -G 'roles:web' test.ping

Compound Matching

1
2
salt -C 'G@os:AlmaLinux and web-*' test.ping
salt -C 'G@roles:web or G@roles:api' test.ping

Pillar Matching

1
salt -I 'environment:prod' test.ping

Best Practices

1. Use Version Control

Always keep your Salt states in Git:

1
2
3
4
cd /srv/salt
git init
git add .
git commit -m "Initial Salt configuration"

2. Test Before Applying

Always use test=True first:

1
salt 'web-server' state.apply test=True

3. Use Pillar for Sensitive Data

Never hardcode passwords in states:

1
2
3
4
5
# Bad
mysql_root_password: 'secret123'

# Good - use pillar
mysql_root_password: {{ pillar['mysql']['root_password'] }}

4. Keep States Idempotent

States should be safe to run multiple times:

1
2
3
4
5
6
7
# Good - checks if already installed
nginx: pkg.installed

# Bad - would fail if already installed
install-nginx:
  cmd.run:
    - name: dnf install -y nginx

5. Use Meaningful State IDs

1
2
3
4
5
6
7
8
9
# Good
nginx-package:
  pkg.installed:
    - name: nginx

# Bad
install1:
  pkg.installed:
    - name: nginx

6. Document Your States

1
2
3
4
5
6
7
8
# Configure nginx web server
# This state:
# - Installs nginx
# - Configures virtual host
# - Manages SSL certificates
nginx:
  pkg.installed:
    - name: nginx

7. Use Requisites

Create proper dependencies:

1
2
3
4
5
6
7
nginx-config:
  file.managed:
    - name: /etc/nginx/nginx.conf
    - require:
        - pkg: nginx # File only created after package installed
    - watch_in:
        - service: nginx # Service reloads if config changes

Troubleshooting

Minion Not Connecting

Check minion configuration:

1
cat /etc/salt/minion | grep master

Check network connectivity:

1
2
nc -zv <master-ip> 4505
nc -zv <master-ip> 4506

Check minion logs:

1
journalctl -u salt-minion -f

Restart minion:

1
systemctl restart salt-minion

State Apply Fails

Run in debug mode:

1
salt 'web-server' state.apply -l debug

Check state syntax:

1
salt 'web-server' state.show_sls nginx

Verify pillar data:

1
salt 'web-server' pillar.items

Keys Not Accepting

Delete and re-add key:

1
2
3
4
salt-key -d web-server
# On minion: rm -rf /etc/salt/pki/minion/minion.pem*
# On minion: systemctl restart salt-minion
salt-key -a web-server

Next Steps

Now that you understand the basics:

Explore More State Modules
- file, pkg, service, user, cron, mount
- Check: https://docs.saltproject.io/en/latest/ref/states/all/
Learn Orchestration
- Coordinate multi-step deployments
- Use Salt’s orchestration runner
Try Reactor System
- React to events automatically
- Self-healing infrastructure
Integrate with Cloud Providers
- AWS, Azure, GCP cloud modules
- Provision and configure VMs automatically
Explore GitFS
- Store states in Git repositories
- Automatic deployment from version control

Conclusion

You’ve learned:

✅ What SaltStack is and why it’s useful
✅ Core concepts: States, Pillar, Grains
✅ How to set up master and minions
✅ Creating and applying states
✅ Remote execution
✅ Targeting minions
✅ Best practices

SaltStack is a powerful tool that can transform how you manage infrastructure. Start small, practice with simple states, and gradually build more complex configurations.

Resources

Official Documentation: https://docs.saltproject.io
Salt Formulas (pre-built states): https://github.com/saltstack-formulas
Community: https://saltproject.io/community/
Salt User Groups: https://www.meetup.com/pro/salt-user-groups/

Happy automating! 🚀

Published on Citizix.com - Your source for DevOps and infrastructure automation guides.