Nvidia Chat with RTX: this local AI tool is very powerful, but it’ll eat a lot of storage

Get your PC ready for it.

3 min. read

Published onFebruary 14, 2024

published onFebruary 14, 2024

Share this article

Read our disclosure page to find out how can you help Windows Report sustain the editorial teamRead more

Nvidia released Chat with RTX, an AI tool that runs locally on Windows PCs, including Windows 11, and while this is just an early demo version, it is a serious competition to the other similar AI tools out there, including Microsoft Copilot.

Chat with RTX runs locally on the Windows PC, meaning it doesn’t need to connect to the Internet to work, and it doesn’t rely on Cloud servers either. The tool lets Windows users choose from two LLMs (Lllama and Mistral), and upload different documents in different formats (docs, videos, notes, or other data), and then it will process these documents according to the user’s requests.

Chat With RTX is a demo app that lets you personalize a GPT large language model (LLM) connected to your own content—docs, notes, videos, or other data. Leveragingretrieval-augmented generation (RAG),TensorRT-LLM, and RTX acceleration, you can query a custom chatbot to quickly get contextually relevant answers. And because it all runs locally on your Windows RTX PC or workstation, you’ll get fast and secure results.

However, being a locally run AI, Chat with RTX will need to be installed on your Windows PC first, and unlikeMicrosoft Copilotor Open AI’s ChatGPT, which can be run online, this tool will require some serious power to run.

You’ll need at least 8GB of VRAM, and 16GB of RAM, and it only runs on Windows 11, for now, as you can see in its system requirements below:

But this is not enough, though. The Demo version of the Chat with RTX requires 35.1 GB of storage to be installed, so prepare to free some space for it. It’s worth mentioning that this is just a demo, so the actual app will most likely require more space, and more power altogether.

We weren’t able to test it out, just yet, but being a locally run AI, although it comes with some advantages, it will have a lot of disadvantages, as well: notably, these kind of AI tools don’t remember context, as it can’t be stored in the cloud, so users might find it frustrating.

However, considering this is just a demo, maybe Nvidia will somehow twitch the model to create a local folder where it can store the context.

We’ll see. But it can be a good option for those who wantsomething differentfrom the regular Copilot or ChatGPT. However, get ready: Nvidia Chat with RTX will use your PC’s resources.

You can download it fromhere.

With KB5043178 to Release Preview Channel, Microsoft advises Windows 11 users to plug in when the battery is low#

Copilot in Outlook will generate personalized themes for you to customize the app#

Microsoft will raise the price of its 365 Suite to include AI capabilities#

Death Stranding Director’s Cut is now Xbox X|S at a huge discount#

Outlook will let users create custom account icons so they can tell their accounts apart easier#

Nvidia Chat with RTX: this local AI tool is very powerful, but it’ll eat a lot of storage#

With KB5043178 to Release Preview Channel, Microsoft advises Windows 11 users to plug in when the battery is low

Copilot in Outlook will generate personalized themes for you to customize the app

Microsoft will raise the price of its 365 Suite to include AI capabilities

Death Stranding Director’s Cut is now Xbox X|S at a huge discount

Outlook will let users create custom account icons so they can tell their accounts apart easier

Nvidia Chat with RTX: this local AI tool is very powerful, but it’ll eat a lot of storage