Model Inference

In EvaDB, every model is a function. We can compose SQL queries using functions as building units similar to conventional SQL functions. EvaDB’s cascades optimizer will optimize the evaluation of user-defined functions for lower latency. Go over Optimizations for more details.

Note

EvaDB ships with a variety of builtin user-defined functions. Go over Models to check them. Did not find the desired model? Go over Functions to create your own user-defined functions and contribute to EvaDB.

1. Projection

The most common usecases are model inference in projections. For example, we can use the MnistImageClassifier to identify numbers from the MINST video.

SELECT MnistImageClassifier(data).label FROM minst_vid;

2. Selection

Another common usecases are model inference in selections. In the below example, we use TextSummarizer and TextClassifier from HuggingFace to summarize the negative food reviews.

SELECT TextSummarizer(data)
FROM food_reviews
WHERE TextClassifier(data).label = 'NEGATIVE';

EvaDB also provides specialized array operators to construct queries. Go over built-in utility operators and functions for all of them. Below is an example of CONTAIN:

SELECT id FROM camera_videos
WHERE ObjectDetector(data).labels @> ['person', 'car'];

3. Lateral Join

In EvaDB, we can also use models in joins. The most powerful usecase is lateral join combined with UNNEST, which is very helpful to flatten the output from one-to-many models. The key idea here is a model could give multiple outputs (e.g., bounding box) stored in an array. This syntax is used to unroll elements from the array into multiple rows. Typical examples are face detectors and object detectors. In the below example, we use emotion detector to detect emotions from faces in the movie, where a single scene can contain multiple faces.

SELECT EmotionDetector(Crop(data, Face.bbox))
FROM movie
LATERAL JOIN UNNEST(FaceDetector(data)) AS Face(bbox, conf);

4. Aggregate Functions

Models can also be executed on a sequence of frames, particularly for action detection. This can be accomplished by utilizing GROUP BY and SEGMENT to concatenate consecutive frames into a single segment.

SELECT ASLActionRecognition(SEGMENT(data))
FROM ASL_ACTIONS
SAMPLE 5
GROUP BY '16 frames';

Here is another example grouping paragraphs from PDFs:

SELECT SEGMENT(data) FROM MyPDFs GROUP BY '10 paragraphs';

5. Order By

Models (typically feature extractors) can also be used in the ORDER BY for embedding-based similarity search. EvaDB also has index support to facilitate this type of queries. In the below examples, we use the SentenceFeatureExtractor to find relevant context When was the NATO created from a collection of pdfs as the knowledge base. Go over PrivateGPT notebook for more details.

SELECT data FROM MyPDFs
ORDER BY Similarity(
    SentenceFeatureExtractor('When was the NATO created?'),
    SentenceFeatureExtractor(data)
);

We can also use the SiftFeatureExtractor to find similar images from a collection of images as the gallery. Go over Image Search for more details.

SELECT name FROM reddit_dataset
ORDER BY Similarity(
    SiftFeatureExtractor(Open('reddit-images/cat.jpg')),
    SiftFeatureExtractor(data)
);

Note

Go over our Usecases to check more ways of utlizing models in real-world use cases.