Pipeline Concepts consist of basic concepts that would be useful to understand how pipelines work and how pipelines can be utilized to get the best out of it.
Following is a visualization of how pipelines are executed and how they go from one stage to another.
In the above image, the stages can access the global context and modify it. However, the stages with [async] can
get/add the context which means they cannot modify the already existing values in the context.
Moreover, the context contains
envs during initialization and other fields can be added to them in other stages.
Notice that the
QueryTranslate step has a check to see if it should continue when an error occurs or not. This is default behaviour and is executed in all stages but is only shown in the case of
queryTranslate to demonstrate how it works.
This behaviour can be tweaked with the
Following pages explain the concepts in detail with examples:
- Pipeline Route Matching
- Pass data between Stages
- Pass inputs to Stage
- Run Stage Asychronously
- Wait for Other Stages
- Write to Global Context
- Pass Environments to Stage
- Error Handling in Pipelines
- Pre-Defined vs User-Defined Stages
- Global Environments
ReactiveSearch Pipelines comes with a lot of features aimed at developers along with observability.
ReactiveSearch provides access to real time logs for pipelines along with features like per stage diffs that shows what exactly happened in each stage. This also helps in checking which stage threw an error and what the error was. How much time did a particular stage take. Everything is provided through logs.
Since Pipelines are aimed at developer first, it is necessary for developers to be able to test them out without affecting production. We provide exactly that, a simulation where the pipeline is executed through the exact same handler that executes the production pipeline but as a mock!
Pipelines can also be edited through our dashboard directly without worrying about adding a certain file to the form data request. Moreover, JS errors are reported with line numbers so that it is easy for developers to debug it.
We have curated a list of how-to guides that would be helpful to get started with ReactiveSearch Pipelines.