Introducing Ntropy AI: your vertical specific LLM orchestration stack

Today we are excited to announce the launch of Ntropy AI, our LLM orchestration platform that allows anyone to run high-throughput, high-complexity tasks 1000x faster and cheaper. 

We built this solution for ourselves, to be able to serve our customers in financial services with the best models for cleaning, enriching and categorizing financial data. 

Without this infrastructure, using GPT4-size models to process 100Ms transactions per day at under 500ms latency would be impossible.

At Ntropy, we are excited to make this possible for every developer across many more tasks. 

From SLM-s and transaction categorization to LLM infrastructure

In January 2021 we set out to solve financial intelligence. We started by building the most accurate models for understanding financial data across different sources, formats and languages.

The last decade of innovation has been around building efficient rails for moving money from point A to point B, instantaneously and very cheaply. 

However, the data derived from money movement is low-quality and fragmented, due to legacy rails and heterogenous formats and standards. This makes financial processes operationally expensive and slow , from onboarding to underwriting, from financial management to accounting and reconciliation. 

With language models, we saw a real opportunity to change that by enabling machines to parse and enrich financial data at scale, in milliseconds with human accuracy, yet at a fraction of the cost. All our models were built in-house by us, end to end.

In 2023 everything changed. GPT4, trained on the internet and not specialized for financial data, beat our specialized small-language-models across many hard benchmarks, without any fine-tuning, zero shot. 

Yet, directly serving GPT4 or any model of that size is not viable. The cost and latency are about 3-4 orders of magnitude too high, even with the latest drops. 

Ntropy 2.0 

Over the last twelve months we have been focused on solving inference cost and latency for our own task, without sacrificing accuracy and reliability. While working on that, we realized that we are onto something much bigger. We have created the fastest and most cost-efficient way to run LLM-s on enterprise tasks.

This infrastructure is already in production with some of our largest customers.

Since the first release, we have had increasing demand from new enterprises across finance and other verticals to leverage it for running LLMs at scale.

Today we are launching our very own LLM orchestration framework, Ntropy AI, to meet this demand. 

Technical details and benchmarks will follow in our next post that is coming out tomorrow. 

Why Ntropy

We are in the early innings of LLM adoption. Accuracy, latency, reliability, cost, and data security remain barriers to entry. To solve for these fundamentals, the options available today are sub-optimal. 

We reduce p99 latencies by 500x lower and average cost per LLM query by 1000x with better accuracy. If you are building in the space and have high volumes , do reach out.

Thank you to our customers, our team and our investors for sticking by us and building with us. Now we can deliver best in class financial intelligence to any application at a cost and speed that no one else can match. 

We can also start executing on the dream of bringing the cost of all intelligence down to zero across any task anywhere.  

This is a significant milestone on Ntropy’s original mission.

I have never been this excited. Cannot wait to see what you get to build.