Andrejus Baranovski

Subscribe to Andrejus Baranovski feed
Blog about Oracle, Full Stack, Machine Learning and Cloud
Updated: 3 hours 46 min ago

Sparrow Apple MLX Backend on Mac Mini M4 (Qwen2 72B 4bit)

Tue, 2024-12-03 09:12
I show how I’m running the Qwen2 72B 4bit model locally on a Mac Mini M4 for Sparrow’s backend. MLX (and MLX-VLM) is the main platform I’m using for local data extraction in Sparrow. 

 

Batch Inference with Qwen2 Vision LLM (Sparrow)

Mon, 2024-11-25 02:38
I'm explaining several hints how to optimize Qwen2 Visual LLM performance for batch processing. 

 

Visual LLM Structured Output Validation with Sparrow

Sun, 2024-11-17 09:24
I explain how Sparrow validates the structured output of visual LLMs to ensure it complies with the JSON schema provided in the query. This process helps prevent errors and hallucinations generated by the LLM.

 

Extracting Financial Market Stock Data from Images with Vision LLM

Sun, 2024-11-10 09:01
In this video, I demonstrate how to extract financial market stock data from images using the powerful Vision LLM Qwen2, all within a Gradio interface. This setup allows quick and easy extraction of key stock stats from screenshots and other image-based data sources—perfect for analysts, traders, and finance enthusiasts looking to streamline data processing. Watch to see how this AI tool can simplify your workflow and make stock data analysis faster and more efficient! 

 

Structured Output Example with Sparrow UI Shell

Mon, 2024-11-04 00:52
Structured output is all you need. I deployed a Sparrow demo UI with Gradio to demonstrate the output Sparrow can produce by running a JSON schema query. You can see examples for the Bonds table, Lab results, and Bank statement. 

 

Qwen2-VL Performance Boost

Sun, 2024-10-20 08:29
I share performance-boosting tips based on my experience using Qwen2-VL in production. 

 

Sparrow Parse Vision LLM FastAPI Endpoint

Sun, 2024-10-13 14:20
Sparrow provides an API for accessing the Sparrow Parse agent, allowing you to run document extraction workflows directly from your existing systems. It helps simplify how data is pulled from documents and integrated into your workflows. 

 

Sparrow Parse Invoice Query with Vision LLM

Tue, 2024-10-08 03:30
New Sparrow Agent - Sparrow Parse, works with Qwen2 Vision LLM. 

What it does: 

1. Accepts query with JSON schema, this helps to solve few things at once - provides JSON structure for LLM to generate response, and hints LLM what types to use for each response element 

2. Runs inference on your GPU of choice, either cloud or local GPU 

3. Validates JSON response, based on query schema

Running Qwen2 Vision LLM on Hugging Face ZeroGPU API

Mon, 2024-09-30 07:38
Explaining my experience running Sparrow Parse with Qwen2 Vision LLM inference on Hugging Face ZeroGPU instance. 

 

Document Querying with Qwen2-VL-7B and JSON Output

Sun, 2024-09-15 13:30
In this video, I demonstrate how to perform document queries using Qwen2-VL-7B. By simplifying field names, we streamline the prompts, making them more efficient and reusable across different documents. This approach is similar to running SQL queries on a database, but tailored for language models like Qwen2-VL-7B, with results returned in JSON format. 

 

Table Parsing with Qwen2-VL-7B

Sun, 2024-09-08 14:00
I show how to retrieve structured JSON output from table image using Qwen2-VL-7B. This VLLM performs OCR and data mapping tasks all out of the box, also it can return structured JSON output without use of intermediate frameworks. 

 

Sparrow Parse: Table Data Extraction with Table Transformer and OCR

Sun, 2024-08-18 08:03
I explain how we extract data with Sparrow Parse, using Table Transformer to identify table area and build table structure to be processed by OCR. Sparrow Parse implements additional logic to clear-up and improve (removing noise, merging columns, adjusting rows) table structure generated by Table Transformer.

 

Table Header Extraction with Table Transformer

Sun, 2024-08-11 13:58
Table Transformer model is able to provide table functional analysis. As result we can identify table header area and build cells to enclose each column header. In the next step with crop each cell and read data with OCR. Finally we get structured data for table header column names. 

 

Invoice Table Detection with Table Transformer

Sun, 2024-07-21 14:02
I show how an open-source transformer model from Microsoft for table detection and structure recognition works. The code is integrated into Sparrow Parse and runs on a local CPU. This approach helps to crop the table area first and then get coordinates for the table cells. Each cell can be cropped and text can be extracted with OCR. This allows retaining the original table structure and reporting the result in JSON or CSV formats. The data extraction part is not in this video; this will be the topic for the next video.

 

Sparrow OCR Service with PaddleOCR

Sun, 2024-07-14 01:29
In this video, I demonstrate the latest updates to the Sparrow OCR Service using PaddleOCR. I walk you through the OCR service workflow in Sparrow, showcasing its integration with FastAPI and highlighting the enhanced functionalities brought by the recent PaddleOCR update. Join me to see how you can leverage these powerful tools for efficient OCR processing! 

 

FastAPI Endpoint for Sparrow LLM Agent

Wed, 2024-07-03 03:10
FastAPI Endpoint for Sparrow LLM Agent. I show how FastAPI endpoint is used in Sparrow to run LLM agent functionality from API client. 

 

Sparrow Parse API for PDF Invoice Data Extraction

Sun, 2024-06-23 08:42
I explain how Sparrow Parse API is integrated into Sparrow for data extraction from PDF documents, such as invoices, receipts, etc. 

 

Avoid LLM Hallucinations: Use Sparrow Parse for Tabular PDF Data, Instructor LLM for Forms

Mon, 2024-06-17 07:06
LLMs tend to hallucinate and produce incorrect results for table data extraction. For this reason in Sparrow we are using Instructor structured output for LLM to query form data and Sparrow Parse to process tabular data within the same document in combined approach. 

 

Effective Table Data Extraction from PDF without LLM

Mon, 2024-06-10 00:56
Sparrow Parse helps to read tabular data from PDFs, relying on various libraries, such as Unstructured or PyMuPDF4LLM. This allows us to avoid data hallucination errors often produced by LLMs when processing complex data structures. 

 

Instructor and Ollama for Invoice Data Extraction in Sparrow [LLM, JSON]

Mon, 2024-06-03 07:11
Structured output from invoice document, running local LLM. This works well with Instructor and Ollama.

 

Pages