Cost-Efficient Rust in Practice, Rust Tokyo Meetup | Paul Masurel, Co-Founder

How can Tantivy index the entire English Wikipedia on a single desktop in just a few minutes? How does Quickwit reach a throughput of 1GB/s on a k8s cluster?

Paul Masurel (Co-Founder & CEO of Quickwit) will tell about building an open-source, cloud-native search engine from the ground up in Rust. Watch to learn about the features, design, internals, and challenges of developing Tantivy and Quickwit.

????Thank you Rust Tokyo for organizing the event! Check out their content:
- Their Youtube Channel: https://www.youtube.com/channel/UCK0U2zrJDzqe8WwFYQSck_g
- Their Meetup: https://www.meetup.com/tokyo-rust-meetup/

????️ TIMECODE :

0:00 Meetup Introduction
1:20 Speaker Bio
4:20 Tantivy Search Benchmark
6:40 Plan of Presentation
7:08 Tantivy's Index & Search Side
10:35 What Makes Tantivy Fast?
14:08 Inverted Index
15:42 Data Layout
21:04 Skip Index, Block of Documents, Document Seeking
25:06 How to Seek into the Right Block of Documents
29:54 Rust is low-level optimization friendly
31:30 Different DocSet traits used & DocSet Intersection
34:30 What is so Special about Quickwit's Log Search?
37:45 What makes Quickwit so cost-efficient?
40:55 Shared-Nothing Architecture Pros & Cons
43:31 Network is faster: Time for Shared Disk Architecture
45:58 Healthy System Load
49:00 Q&A Time
___

⭐️ Psst...on GitHub there is a Star button...you know...only if you like our content : https://github.com/quickwit-oss/quickwit

???? Tantivy Repo: https://github.com/quickwit-oss/tantivy

???? Subscribe for more videos about software engineering & architecture : https://www.youtube.com/channel/UCvZV...

???? Join our community :

- Mind-blowing Discord : https://discord.gg/q5fquGeR3D
- Sky-breaking blogs : https://quickwit.io/blog
- Lightning-speed Quickwit installation : https://quickwit.io/docs/
- Latest & greatest news on Twitter : https://twitter.com/Quickwit_Inc
- Cosmos-litting videos: https://www.youtube.com/channel/UCvZVuRm2FiDq1_ul0mY85wA

Topics: indexing, search, data, query, architecture, sre, devops, metastore, metadata, indexer, scaling indexes, cluster membership, search engine internals, devsecops, shards, search engine library, architecture, software architecture

#softwareengineer #rustlang #devops