Master Elasticsearch with Python: From Installation to Advanced Queries

What is Elasticsearch?

Elasticsearch (ES) is a highly available, distributed open‑source search engine built on Apache Lucene. It stores unstructured JSON data, making it a NoSQL database that also provides powerful search capabilities.

Elasticsearch Use Cases

Providing robust search and autocomplete for dynamic content sites such as e‑commerce platforms or blogs.

Collecting and analyzing log data to discover trends and statistics.

Setup and Run

Download the Elasticsearch binary, ensure Java 7+ is installed, unzip, and run the executable. Verify the installation by accessing http://localhost:9200 or using curl to see the welcome page.

Basic Example

Create an index (e.g., company) using a REST client like PostMan. The index stores documents similar to a database table. Access http://localhost:9200/company to view its settings, including mappings, shards, and replicas.

Insert a document by sending JSON via POST; Elasticsearch assigns an _id if none is provided. Retrieve the document with GET /company/employees/1 or view only the source with GET /company/employees/1/_source. Update a document by sending a modified JSON payload, and delete documents with the DELETE method.

Searching

Perform a basic search with GET /company/employees/_search?q=adnan, which returns matching records and a max_score indicating relevance. Restrict searches to specific fields, e.g., q=name:Adnan, analogous to an SQL SELECT statement.

Using Elasticsearch in Python

The REST API can be accessed via the requests library, but the official elasticsearch Python client simplifies interactions. Install it with pip install elasticsearch and verify the installation by running a short script.

Web Search and Elasticsearch

Example project: scrape recipes from Allrecipes, store them in an index named recipes with a type salads, and define strict mappings for fields such as title (text) and calories (integer).

Steps include:

Fetching JSON data from the web.

Connecting to Elasticsearch ( es.ping()).

Creating the index with mappings, using ignore=400 to avoid overwriting existing indices.

Indexing documents, handling errors when field types mismatch (e.g., inserting a string into an integer field).

Query the indexed data with functions that accept an index name and search criteria, demonstrating filters such as calories=102 or calories>20, and selecting specific fields in the response.

Conclusion

Elasticsearch is a powerful tool for building searchable applications, offering precise result sets and advanced features like fuzzy search. Continue exploring the documentation and consider learning Query DSL for more complex queries.