How to Restore Elasticsearch Snapshot

Elasticsearch snapshots are critical backups of your clusters data, indices, and configuration. Whether youre recovering from hardware failure, accidental deletion, or migrating to a new environment, the ability to restore an Elasticsearch snapshot ensures business continuity and data integrity. Unlike simple file copies, Elasticsearch snapshots are intelligent, incremental, and storage-efficient, leveraging shared segments to minimize disk usage. Restoring a snapshot is not merely a technical procedureits a strategic operation that demands precision, planning, and understanding of your clusters state. This guide provides a comprehensive, step-by-step walkthrough on how to restore Elasticsearch snapshots, covering everything from prerequisites to advanced scenarios, best practices, real-world examples, and common pitfalls. By the end, youll have the confidence to restore snapshots reliably in any production or development context.

Step-by-Step Guide

Prerequisites Before Restoration

Before initiating any snapshot restoration, ensure your environment meets the following requirements:

• Elasticsearch version compatibility: Snapshots can only be restored to the same major version or a higher version of Elasticsearch. For example, a snapshot taken on Elasticsearch 7.10 can be restored on 7.17 or 8.x, but not on 6.x.
• Access to the snapshot repository: The repository where the snapshot is stored (e.g., S3, HDFS, NFS, or Azure Blob) must be accessible and properly configured on the target cluster.
• Sufficient disk space: Ensure the target cluster has adequate storage to accommodate the restored indices. Snapshot sizes are compressed, but restored data may expand significantly.
• Cluster health: The target cluster should be in a healthy state (green or yellow) with no ongoing shard allocation issues.
• Index settings and mappings compatibility: If restoring to a cluster with different settings (e.g., number of shards, analysis chains), verify that the target cluster can support the restored index configurations.

Failure to meet these prerequisites may result in restoration failures, incomplete data, or cluster instability.

Step 1: List Available Snapshots

Before restoring, identify which snapshots are available in your repository. Use the following API call:

GET /_snapshot/my_backup_repository/_all

Replace my_backup_repository with the actual name of your snapshot repository. This returns a JSON response listing all snapshots in the repository, including:

• snapshot: The name of the snapshot
• version: The Elasticsearch version used to create the snapshot
• state: The status (e.g., SUCCESS, FAILED, IN_PROGRESS)
• start_time: When the snapshot was initiated
• end_time: When the snapshot completed
• indices: List of indices included in the snapshot

Example response snippet:

{
"snapshots": [
{
"snapshot": "daily_backup_20240501",
"version": "8.12.0",
"state": "SUCCESS",
"start_time": "2024-05-01T02:00:00.000Z",
"end_time": "2024-05-01T02:15:30.000Z",
"indices": [
"logs-2024-04",
"user_profiles",
"metrics"
]
}
]
}

Use this information to select the correct snapshot for restoration. If you need to restore a specific index from a snapshot containing multiple indices, note its name precisely.

Step 2: Verify Repository Configuration

Ensure the snapshot repository is correctly registered on the target cluster. Use the following API to list all registered repositories:

GET /_snapshot/_all

If your desired repository (e.g., my_backup_repository) does not appear, you must register it. For example, to register an S3 repository:

PUT /_snapshot/my_backup_repository
{
"type": "s3",
"settings": {
"bucket": "my-elasticsearch-backups",
"region": "us-west-2",
"base_path": "snapshots/",
"access_key": "your-access-key",
"secret_key": "your-secret-key"
}
}

For NFS or shared filesystem repositories:

PUT /_snapshot/my_nfs_repo
{
"type": "fs",
"settings": {
"location": "/mnt/elasticsearch/snapshots",
"compress": true
}
}

Always validate repository access by attempting a test snapshot or listing its contents. Misconfigured repositories are a leading cause of restoration failures.

Step 3: Close or Delete Conflicting Indices (If Necessary)

If you are restoring an index that already exists in the target cluster, Elasticsearch will reject the restore operation by default. You have two options:

1. Close the existing index: This preserves the index structure and settings while making it available for restoration.
2. Delete the existing index: Removes all data and allows a clean restore.

To close an index:

POST /logs-2024-04/_close

To delete an index:

DELETE /logs-2024-04

Use caution when deleting. Always confirm you have a current backup or can afford to lose the existing data. Closing is safer for temporary conflicts, especially if you plan to restore and then reopen the index.

Step 4: Initiate the Restore Operation

Once prerequisites are met, initiate the restore using the _restore API. The simplest form restores all indices in the snapshot:

POST /_snapshot/my_backup_repository/daily_backup_20240501/_restore

This command restores all indices in the snapshot with their original names and settings. Elasticsearch will begin allocating shards and copying data. You can monitor progress using:

GET /_cat/tasks?v&detailed=true

Look for tasks with action containing snapshot/restore to track the operation.

Step 5: Restore Specific Indices (Optional)

You may not want to restore all indices from a snapshot. To restore only specific indices, use the indices parameter:

POST /_snapshot/my_backup_repository/daily_backup_20240501/_restore
{
"indices": "logs-2024-04,metrics",
"ignore_unavailable": true,
"include_global_state": false
}

• indices: Comma-separated list of index names to restore.
• ignore_unavailable: If set to true, the restore will proceed even if some specified indices dont exist in the snapshot.
• include_global_state: Set to false unless you need to restore cluster-wide settings (e.g., templates, ingest pipelines, keystore entries). Restoring global state can overwrite existing configurations.

Restoring selective indices is useful for disaster recovery scenarios where only a subset of data is affected.

Step 6: Rename Indices During Restore

One of the most powerful features of Elasticsearch snapshot restoration is the ability to rename indices during the process. This is essential for testing, migration, or avoiding naming conflicts.

To rename indices, use the rename_pattern and rename_replacement parameters:

POST /_snapshot/my_backup_repository/daily_backup_20240501/_restore
{
"indices": "logs-2024-04",
"rename_pattern": "logs-(.+)",
"rename_replacement": "logs-2024-04-test-$1"
}

In this example:

• logs-2024-04 is renamed to logs-2024-04-test-2024-04
• The regex group (.+) captures everything after logs-
• $1 inserts the captured group into the new name

This technique is invaluable for creating test environments, performing A/B comparisons, or safely validating backups before full restoration.

Step 7: Monitor Restore Progress

Restoration can take minutes to hours depending on data volume, network speed, and cluster resources. Monitor the process using:

GET /_cat/indices?v

Look for indices in a yellow or green state. Yellow indicates unassigned replicas (common during restore), while green means all shards are allocated.

For detailed task progress:

GET /_tasks?detailed=true&actions=*restore*

This returns detailed information such as:

• Percentage completed
• Number of files transferred
• Bytes transferred
• Estimated time remaining

Additionally, check cluster health:

GET /_cluster/health?pretty

Wait until the cluster status returns green before proceeding with queries or production traffic.

Step 8: Reopen Indices and Verify Data

If you closed indices before restoration, reopen them:

POST /logs-2024-04/_open

Verify the restored data by querying the index:

GET /logs-2024-04/_search
{
"size": 1
}

Check document count:

GET /logs-2024-04/_count

Compare the count with the original source or backup metadata to confirm completeness. Validate mappings and settings using:

GET /logs-2024-04/_mapping
GET /logs-2024-04/_settings

Ensure analyzers, field types, and replication settings match expectations. If using ingest pipelines, test document ingestion to confirm they are properly restored.

Best Practices

Test Restorations Regularly

Many organizations assume their snapshots are valid because they complete without error. However, a snapshot that appears successful may still be unusable due to corruption, incompatible settings, or missing dependencies. Establish a routine of restoring snapshots to a non-production cluster at least quarterly. Document the process and time required. This ensures your backup strategy is viable when disaster strikes.

Use Version-Specific Snapshots

Never attempt to restore a snapshot from a newer major version to an older one. Elasticsearch does not support backward compatibility for snapshots. Always maintain a snapshot repository per major version. For example, maintain separate repositories for 7.x, 8.x, etc.

Enable Compression and Use Incremental Snapshots

By default, Elasticsearch compresses snapshots using the LZF algorithm. Ensure compression is enabled in your repository settings:

"compress": true

Additionally, Elasticsearch snapshots are inherently incremental. Only new or changed segments are uploaded in subsequent snapshots. This reduces storage costs and speeds up creation. Avoid deleting old snapshots unless absolutely necessaryElasticsearch relies on shared segments for efficient restoration.

Limit Concurrent Restores

Restoring multiple snapshots simultaneously can overwhelm cluster resources, leading to timeouts, shard failures, or node crashes. Limit concurrent restore operations to one or two at a time, especially on clusters with limited memory or disk I/O. Use task monitoring to ensure one restore completes before initiating another.

Separate Snapshot Repositories by Purpose

Use distinct repositories for different use cases:

• daily-backups: For routine operational backups
• pre-migration: For snapshots taken before major upgrades
• test-repo: For development and QA validation

This prevents accidental overwrites and simplifies recovery workflows.

Automate Snapshot Lifecycle

Use Elasticsearchs Index Lifecycle Management (ILM) or Curator (for older versions) to automate snapshot creation and retention. For example, create a snapshot daily and retain only the last 30. Automate cleanup of expired snapshots to prevent storage bloat.

Document Your Snapshot Strategy

Document:

• Which indices are included in each snapshot
• Retention policies
• Repository locations and access credentials
• Restoration procedures and expected downtime

Store this documentation in a version-controlled repository (e.g., Git) and make it accessible to operations and DevOps teams.

Validate Data Integrity Post-Restore

After restoration, run checksums or compare document counts, field distributions, and date ranges between the source and restored data. Use Kibanas Discover tab or custom scripts to validate data quality. Dont assume completenessverify it.

Plan for Large Restores

For snapshots exceeding 100 GB, plan for extended restoration times. Schedule during maintenance windows. Consider increasing:

• thread_pool.search.size: To handle more concurrent shard recovery
• indices.recovery.max_bytes_per_sec: To increase network throughput (default is 40MB/s)

Example adjustment in elasticsearch.yml:

indices.recovery.max_bytes_per_sec: 200mb

Restart the cluster after changing these settings.

Tools and Resources

Elasticsearch Snapshot API

The primary interface for managing snapshots is the REST API. Key endpoints include:

• GET /_snapshot List repositories
• PUT /_snapshot/{repository} Create repository
• GET /_snapshot/{repository}/_all List snapshots
• POST /_snapshot/{repository}/{snapshot}/_restore Restore snapshot
• DELETE /_snapshot/{repository}/{snapshot} Delete snapshot

Always refer to the official Elasticsearch documentation for your version: Elasticsearch Snapshots Guide.

Kibana Snapshot and Restore UI

For users who prefer a graphical interface, Kibanas Stack Management > Snapshot and Restore section provides a visual interface to:

• View registered repositories
• List available snapshots
• Initiate restores with a form-based interface
• Monitor restore progress

While the UI is user-friendly, it lacks advanced options like index renaming and fine-grained control. Use it for basic operations, but rely on the API for production-critical restores.

Curator (Legacy Tool)

For Elasticsearch versions prior to 7.0, Curator (a Python-based tool) was widely used to automate snapshot creation and deletion. While largely superseded by ILM, Curator remains useful for legacy systems. Install via pip:

pip install elasticsearch-curator

Configure via YAML files to define snapshot policies and retention rules.

Third-Party Tools

Several third-party tools enhance snapshot management:

• Elastic Cloud: Fully managed snapshots with automatic retention and cross-region replication.
• Opsters Snapshot Manager: Open-source tool that provides enhanced monitoring, alerting, and automation for snapshot operations.
• Logstash + Filebeat: While not for snapshots, these tools help ensure continuous data ingestion so snapshots remain up-to-date.

Monitoring and Alerting

Use Elasticsearchs built-in monitoring features or integrate with Prometheus and Grafana to track:

• Snapshots that fail or remain in progress
• Repository disk usage
• Cluster health during restore

Set up alerts for:

• Snapshots older than 24 hours
• Repository storage usage exceeding 80%
• Restore operations taking longer than 2 hours

Documentation and Community

Key resources:

• Official Elasticsearch Snapshot Documentation
• Elastic Discuss Forum
• Elasticsearch GitHub Repository
• Elastic Blog: Snapshot Best Practices

Real Examples

Example 1: Restoring a Production Index After Accidental Deletion

A developer accidentally ran DELETE /customer_data in production. The index contained 12 million documents and was critical for customer onboarding.

Response:

1. Checked available snapshots: GET /_snapshot/prod_repo/_all
2. Found a snapshot from 2 hours ago: prod_daily_20240501
3. Confirmed the index customer_data was included in the snapshot.
4. Executed restore with rename to avoid conflict:

POST /_snapshot/prod_repo/prod_daily_20240501/_restore
{
"indices": "customer_data",
"rename_pattern": "customer_data",
"rename_replacement": "customer_data_restored"
}

5. Monitored restore progress for 45 minutes.
6. Verified document count: GET /customer_data_restored/_count returned 12,000,487.
7. Reindexed data back to original name using Reindex API:

POST /_reindex
{
"source": {
"index": "customer_data_restored"
},
"dest": {
"index": "customer_data"
}
}

8. Deleted the temporary index: DELETE /customer_data_restored

Result: Full data recovery with zero downtime to end users.

Example 2: Migrating Data Between Clusters

A company is migrating from an on-premises Elasticsearch 7.17 cluster to an AWS Elasticsearch 8.12 cluster.

Process:

1. Created an S3 repository on the source cluster.
2. Taken a full snapshot: daily_migration_20240501.
3. Registered the same S3 repository on the target cluster with IAM credentials.
4. Verified snapshot visibility: GET /_snapshot/s3_repo/daily_migration_20240501.
5. Restored all indices with index renaming to avoid naming conflicts:

POST /_snapshot/s3_repo/daily_migration_20240501/_restore
{
"rename_pattern": "(.+)",
"rename_replacement": "migrated_$1"
}

6. Updated Kibana dashboards to point to new index names.
7. Reconfigured Logstash pipelines to write to new indices.
8. Verified data integrity with sample queries and comparison scripts.
9. Decommissioned the old cluster after 72 hours of validation.

Result: Seamless migration with no data loss and minimal service disruption.

Example 3: Restoring a Single Index from a Multi-Index Snapshot

A team needs to restore only the error_logs index from a daily snapshot containing 15 indices.

Process:

1. Identified the snapshot: daily_full_20240501
2. Executed selective restore:

POST /_snapshot/full_repo/daily_full_20240501/_restore
{
"indices": "error_logs",
"include_global_state": false
}

3. Restored index appeared as error_logs (no rename needed).
4. Verified document count matched source.
5. Confirmed ingest pipeline and index template were correctly applied.

Result: Reduced restore time from 2 hours to 12 minutes by avoiding unnecessary data transfer.

FAQs

Can I restore a snapshot to a different Elasticsearch version?

You can only restore a snapshot to the same major version or a higher one. For example, a snapshot from Elasticsearch 7.16 can be restored on 7.17 or 8.x, but not on 6.x or 5.x. Downgrading is not supported.

What happens if a snapshot is corrupted during restoration?

Elasticsearch performs checksum validation during restore. If a segment is corrupted, the restore will fail with an error message indicating the corrupted file. Youll need to restore from an earlier, valid snapshot.

Do snapshots include index templates and ingest pipelines?

By default, snapshots do not include cluster-wide settings like index templates, ingest pipelines, or security configurations. To include them, set "include_global_state": true during restore. Use this option cautiously, as it can overwrite existing configurations.

How long does a snapshot restoration take?

Restoration time depends on:

• Size of the snapshot
• Cluster disk I/O and network bandwidth
• Number of shards
• Node count and hardware

As a rule of thumb: 10 GB takes 510 minutes; 100 GB takes 3060 minutes; 1 TB may take several hours.

Can I restore a snapshot while the cluster is under load?

Yes, but its not recommended. Restoration consumes significant I/O and network resources. Perform restores during low-traffic periods to avoid performance degradation or timeouts.

Whats the difference between restoring and reindexing?

Restoring a snapshot is faster and preserves all metadata (settings, mappings, aliases). Reindexing copies data from one index to another but requires reapplying settings and mappings manually. Use restore for full recovery; use reindex for transformation or migration between differently configured indices.

Can I restore a snapshot to a cluster with fewer nodes?

Yes, but Elasticsearch will adjust shard allocation. If the snapshot contains 5 primary shards and your target cluster has only 2 nodes, shards will be redistributed across those nodes. Ensure sufficient disk space and memory per node to handle the increased load.

How do I delete a snapshot to free up space?

Use the DELETE API:

DELETE /_snapshot/my_repo/snapshot_name

Elasticsearch automatically removes unused segments from the repository, making deletion space-efficient.

Why is my restored index in yellow state?

A yellow state means all primary shards are allocated, but replica shards are not. This is normal during restoration. Once the cluster stabilizes and resources allow, replicas will be assigned automatically. If it remains yellow, check cluster health, disk space, or shard allocation settings.

Is it safe to delete the original index before restoring?

Only if you are certain you no longer need the original data. Always confirm the snapshot is valid and accessible before deletion. Use index closing as a safer alternative if you need to preserve the index structure.

Conclusion

Restoring an Elasticsearch snapshot is a fundamental skill for any team managing data at scale. It is not a simple copy and paste operationit requires understanding of cluster state, version compatibility, storage architecture, and recovery workflows. By following the step-by-step guide in this tutorial, youve learned how to identify, prepare, execute, and validate snapshot restorations with precision. Youve explored best practices that prevent common pitfalls, tools that enhance automation, real-world scenarios that demonstrate practical application, and answers to frequently asked questions that clarify ambiguity.

Remember: A backup is only as good as its restore. Regularly test your snapshots. Document your procedures. Monitor your repositories. Automate retention. And never assumealways verify.

With this knowledge, you are now equipped to recover your Elasticsearch data confidently, whether in the face of accidental deletion, system failure, or planned migration. The resilience of your data infrastructure begins with a well-executed restore. Make it part of your operational DNA.