convex-jina

Convert any URL to clean markdown and search the web with Jina AI in Convex, featuring reactive queries and durable caching.

Installation

npm install convex-jina

About convex-jina

Read any URL as clean markdown and search the web - with durable caching and reactive queries, powered by Jina AI.

Benefits

Use cases

how to scrape web pages in Convex application

Use convex-jina to read any URL and convert it to clean markdown within your Convex functions. The component handles the HTTP requests to Jina AI Reader API and provides durable caching so repeated requests don't hit rate limits or incur additional costs.

add web search to Convex backend

The convex-jina component wraps Jina AI Search API to perform web searches directly from your Convex mutations and queries. Results are cached and can be accessed reactively, enabling real-time search features in your application.

cache external API responses in Convex

convex-jina automatically caches all Jina AI API responses in your Convex database. This prevents duplicate API calls for the same URL or search query, reducing costs and improving response times for your users.

Frequently asked questions

Does convex-jina work with any website URL?

convex-jina uses Jina AI Reader API which can process most publicly accessible websites and convert them to clean markdown. However, some sites with heavy JavaScript rendering or access restrictions may not work perfectly. The component handles errors gracefully and provides fallback mechanisms.

How does caching work with convex-jina?

convex-jina stores all API responses in your Convex database with configurable TTL. When you request the same URL or search query, it returns cached results instead of making new API calls. This reduces costs and improves performance while keeping data fresh through automatic expiration.

Can I use convex-jina for real-time web content monitoring?

Yes, convex-jina supports reactive queries through Convex's subscription system. You can set up queries that automatically update your frontend when cached web content changes or when new search results are available, enabling real-time monitoring dashboards.

What Jina AI APIs does convex-jina support?

convex-jina wraps both Jina AI Reader API for converting URLs to markdown and Jina AI Search API for web search functionality. Both APIs are integrated with Convex's reactive system and include built-in caching and error handling.

Links