TOPTRACING

Lobe Chat: an open-source, modern-design AI chat framework

Dec 6, 2024

0
https://lobehub.com/https://github.com/lobehub/lobe-chat

Lobe Chat is an open-source, modern-design ChatGPT/LLMs UI/Framework. Supports speech-synthesis, multi-modal, and extensible (function call) plugin system. One-click FREE deployment of your private OpenAI ChatGPT/Claude/Gemini/Groq/Ollama chat application.

Build your personal AI assistant and professional team, and together with AI, propel your creative ventures, writing projects, learning journeys, and career tasks. Henceforth, command a specialized squad to tackle specific challenges and enhance work efficiency, standing out in the era of individual ascendancy. This will become a reality in LobeChat.

Features

Model Vision Recognition

LobeChat now supports large language models with visual recognition capabilities such as OpenAI's gpt-4-vision, Google Gemini Pro vision, and Zhipu GLM-4 Vision, enabling LobeChat to have multimodal interaction capabilities. Users can easily upload or drag and drop images into the chat box, and the assistant will be able to recognize the content of the images and engage in intelligent conversations based on them, creating more intelligent and diverse chat scenarios.

This feature opens up new ways of interaction, allowing communication to extend beyond text and encompass rich visual elements. Whether it's sharing images in daily use or interpreting images in specific industries, the assistant can provide an excellent conversational experience.

TTS & STT Voice Conversation

LobeChat supports Text-to-Speech (TTS) and Speech-to-Text (STT) technologies. Our application can convert text information into clear voice output, allowing users to interact with our conversational agents as if they were talking to a real person. Users can choose from a variety of voices and pair the appropriate audio with the assistant. Additionally, for users who prefer auditory learning or need to obtain information while busy, TTS provides an excellent solution.

In LobeChat, we have carefully selected a series of high-quality voice options (OpenAI Audio, Microsoft Edge Speech) to meet the needs of users from different regions and cultural backgrounds. Users can choose suitable voices based on personal preferences or specific scenarios, thereby obtaining a personalized communication experience.

Lobe TTS

@lobehub/tts is a high-quality TTS toolkit developed using the TS language, supporting usage in both server and browser environments.

  • Server: With just 15 lines of code, it can achieve high-quality speech generation capabilities comparable to OpenAI TTS services. It currently supports EdgeSpeechTTS, MicrosoftTTS, OpenAITTS, and OpenAISTT.
  • Browser: It provides high-quality React Hooks and visual audio components, supporting common functions such as loading, playing, pausing, and dragging the timeline, and offering extensive audio track style adjustment capabilities.

Text to Image Generation

Supporting the latest text-to-image generation technology, LobeChat now enables users to directly utilize the Text to Image tool during conversations with the assistant. By harnessing the capabilities of AI tools such as DALL-E 3, MidJourney, and Pollinations, assistants can now transform your ideas into images. This allows for a more private and immersive creative process.

Plugin System

The plugin ecosystem of LobeChat is an important extension of its core functionality, greatly enhancing the practicality and flexibility of the LobeChat assistant.

By utilizing plugins, LobeChat assistants can obtain and process real-time information, such as searching for web information and providing users with instant and relevant news.

In addition, these plugins are not limited to news aggregation, but can also extend to other practical functions, such as quickly searching documents, generating images, obtaining data from various platforms like Bilibili, Steam, and interacting with various third-party services.

Learn more about plugin usage by checking it out.

Plugin Ecosystem

  • lobe-chat-plugins: This is the plugin index for LobeChat. It retrieves the list of plugins from the index.json of this repository and displays them to the users.
  • chat-plugin-template: Chat Plugin plugin development template, you can quickly create a new plugin project through the project template.
  • @lobehub/chat-plugin-sdk: The LobeChat plugin SDK can help you create excellent Lobe Chat plugins.
  • @lobehub/chat-plugins-gateway: The LobeChat plugin gateway is a backend service that serves as the gateway for LobeChat plugins. We deploy this service using Vercel.

Assistant Market

In LobeChat's Assistant Market, creators can discover a vibrant and innovative community that brings together numerous carefully designed assistants. These assistants not only play a crucial role in work scenarios but also provide great convenience in the learning process. Our market is not just a showcase platform, but also a collaborative space. Here, everyone can contribute their wisdom and share their personally developed assistants.

Multi-Model Service Provider Support

In the continuous development of LobeChat, we deeply understand the importance of diversity in model service providers for meeting the needs of the community when providing AI conversation services. Therefore, we have expanded our support to multiple model service providers, rather than being limited to a single one, in order to offer users a more diverse and rich selection of conversations.

In this way, LobeChat can more flexibly adapt to the needs of different users, while also providing developers with a wider range of choices.

Supported Model Service Providers

We have implemented support for the following model service providers:

  • AWS Bedrock: Integrated with AWS Bedrock service, supporting models such as Claude / LLama2, providing powerful natural language processing capabilities. Learn more
  • Anthropic (Claude): Accessed Anthropic's Claude series models, including Claude 3 and Claude 2, with breakthroughs in multi-modal capabilities and extended context, setting a new industry benchmark. Learn more
  • Google AI (Gemini Pro, Gemini Vision): Access to Google's Gemini series models, including Gemini and Gemini Pro, to support advanced language understanding and generation. Learn more
  • ChatGLM: Added the ChatGLM series models from Zhipuai (GLM-4/GLM-4-vision/GLM-3-turbo), providing users with another efficient conversation model choice. Learn more
  • Moonshot AI (Dark Side of the Moon): Integrated with the Moonshot series models, an innovative AI startup from China, aiming to provide deeper conversation understanding. Learn more
  • Groq: Accessed Groq's AI models, efficiently processing message sequences and generating responses, capable of multi-turn dialogues and single-interaction tasks. Learn more
  • OpenRouter: Supports routing of models including Claude 3, Gemma, Mistral, Llama2 and Cohere, with intelligent routing optimization to improve usage efficiency, open and flexible. Learn more
  • 01.AI (Yi Model): Integrated the 01.AI models, with series of APIs featuring fast inference speed, which not only shortened the processing time, but also maintained excellent model performance. Learn more
  • Together.ai: Over 100 leading open-source Chat, Language, Image, Code, and Embedding models are available through the Together Inference API. For these models you pay just for what you use. Learn more
  • Minimax: Integrated the Minimax models, including the MoE model abab6, offers a broader range of choices. Learn more
  • DeepSeek: Integrated with the DeepSeek series models, an innovative AI startup from China, The product has been designed to provide a model that balances performance with price. Learn more
  • Qwen: Integrated with the Qwen series models, including the latest qwen-turbo, qwen-plus and qwen-max. Learn more

At the same time, we are also planning to support more model service providers, such as Replicate and Perplexity, to further enrich our service provider library. If you would like LobeChat to support your favorite service provider, feel free to join our community discussion.

Local Large Language Model (LLM) Support

With the release of LobeChat v0.127.0, we are excited to introduce a groundbreaking feature - Ollama AI support! 🤯 With the powerful infrastructure of Ollama AI and the community's collaborative efforts, you can now engage in conversations with a local LLM (Large Language Model) in LobeChat! 🤩

We are thrilled to introduce this revolutionary feature to all LobeChat users at this special moment. The integration of Ollama AI not only signifies a significant technological leap for us but also reaffirms our commitment to continuously pursue more efficient and intelligent communication.

How to Start a Conversation with Local LLM?

The startup process is exceptionally simple! By running the following Docker command, you can experience conversations with a local LLM in LobeChat:

docker run -d -p 3210:3210 -e OLLAMA_PROXY_URL=http://host.docker.internal:11434/v1 lobehub/lobe-chat

Yes, it's that simple! 🤩 You don't need to go through complicated configurations or worry about a complex installation process. We have prepared everything for you. With just one command, you can engage in deep conversations with a local AI.

Experience Unprecedented Interaction Speed

With the powerful capabilities of Ollama AI, LobeChat has greatly improved its efficiency in natural language processing. Both processing speed and response time have reached new heights. This means that your conversational experience will be smoother, without any waiting, and with instant responses.

Why Choose a Local LLM?

Compared to cloud-based solutions, a local LLM provides higher privacy and security. All your conversations are processed locally, without passing through any external servers, ensuring the security of your data. Additionally, local processing can reduce network latency, providing you with a more immediate communication experience.

Embark on Your LobeChat & Ollama AI Journey

Now, let's embark on this exciting journey together! Through the collaboration of LobeChat and Ollama AI, explore the endless possibilities brought by AI. Whether you are a tech enthusiast or simply curious about AI communication, LobeChat will offer you an unprecedented experience.

Local / Cloud Database

In modern application development, the choice of data storage solution is crucial. To meet the needs of different users, LobeChat offers flexible configurations that support both local and server-side databases. Whether you prioritize data privacy and control or seek a convenient user experience, LobeChat can provide excellent solutions for you.

Local Database: Data Control and Privacy Protection

For users who prefer more control over their data and value privacy protection, LobeChat offers support for local databases. By using IndexedDB as the storage solution and combining it with dexie as an Object-Relational Mapping (ORM) tool, LobeChat achieves efficient data management.

Additionally, we have introduced Conflict-Free Replicated Data Type (CRDT) technology to ensure a seamless multi-device synchronization experience. This experimental feature aims to provide users with greater autonomy and data security.

Furthermore, we have attempted to introduce CRDT technology to achieve cross-device synchronization based on the local database. This experimental feature aims to provide users with greater autonomy and data security.

Server-Side Database: Convenient and Efficient User Experience

For users who seek a convenient user experience, LobeChat supports PostgreSQL as the server-side database. By managing data with Dirzzle ORM and combining it with Clerk for authentication, LobeChat can offer users an efficient and reliable server-side data management solution.

Server-Side Database Technology Stack

  • DB: PostgreSQL (Neon is the default)
  • ORM: Dirzzle ORM
  • Auth: Clerk
  • Server Router: tRPC

Deployment Solution Selection Guide

1. Local Database

The local database solution is suitable for users who wish to have strict control over their data. With LobeChat's support for local databases, you can securely store and manage data without relying on external servers. This solution is particularly suitable for users with high requirements for data privacy.

2. Server-Side Database

The server-side database solution is ideal for users who want to simplify data management processes and enjoy a convenient user experience. Through server-side databases and user authentication, LobeChat can ensure the security and efficiency of data. If you want to learn how to configure a server-side database, please refer to our detailed documentation.

Whether you choose a local database or a server-side database, LobeChat can provide you with an excellent user experience.

Support Multi-User Management

In modern applications, user management and identity verification are essential functions. To meet the diverse needs of different users, LobeChat provides two main user authentication and management solutions: next-auth and Clerk. Whether you are looking for simple user registration and login or need advanced multi-factor authentication and user management, LobeChat can flexibly accommodate your requirements.

next-auth: Flexible and Powerful Identity Verification Library

LobeChat integrates next-auth, a flexible and powerful identity verification library that supports various authentication methods, including OAuth, email login, and credential login. With next-auth, you can easily achieve the following functions:

  • User Registration and Login: Support various authentication methods to meet different user needs.
  • Session Management: Efficiently manage user sessions to ensure security.
  • Social Login: Support quick login via various social platforms.
  • Data Security: Ensure the security and privacy of user data.

Clerk: Modern User Management Platform

For users requiring advanced user management features, LobeChat also supports Clerk, a modern user management platform. Clerk offers richer functionality to help you achieve higher security and flexibility:

  • Multi-Factor Authentication (MFA): Provides higher security protection.
  • User Profile Management: Conveniently manage user information and configurations.
  • Login Activity Monitoring: Real-time monitoring of user login activities to ensure account security.
  • Scalability: Supports complex user management requirements.

Progressive Web App (PWA)

We understand the importance of providing a seamless experience for users in today's multi-device environment. To achieve this, we have adopted Progressive Web App PWA technology, which is a modern web technology that elevates web applications to a near-native app experience. Through PWA, LobeChat is able to provide a highly optimized user experience on both desktop and mobile devices, while maintaining lightweight and high performance characteristics. Visually and perceptually, we have also carefully designed it to ensure that its interface is indistinguishable from a native app, providing smooth animations, responsive layouts, and adaptation to different screen resolutions of various devices.

Mobile Device Adaptation

LobeChat has undergone a series of optimized designs for mobile devices to enhance the user's mobile experience.

Currently, we are iterating on the user experience for mobile devices to achieve a smoother and more intuitive interaction. If you have any suggestions or ideas, we warmly welcome your feedback through GitHub Issues or Pull Requests.

Custom Themes

LobeChat places a strong emphasis on personalized user experiences in its interface design, and thus introduces flexible and diverse theme modes, including a light mode for daytime and a dark mode for nighttime.

In addition to theme mode switching, we also provide a series of color customization options, allowing users to adjust the application's theme colors according to their preferences. Whether it's a stable deep blue, a lively peach pink, or a professional gray and white, users can find color choices in LobeChat that match their own style.

For users who prefer to manually adjust details, LobeChat also provides intuitive setting options and offers a choice between conversation bubble mode and document mode for chat scenes.