Skip to content

Latest commit

 

History

History
97 lines (75 loc) · 4.34 KB

README.md

File metadata and controls

97 lines (75 loc) · 4.34 KB
Raw

Native LangChain4j Searchbot

This is a Java command-line chatbot example application that leverages the LangChain4j API and version GPT-4 of OpenAI.

LangChain4j is a Java implementation of LangChain whose goal is to simplify integrating AI/LLM capabilities into Java applications.

The example is based on a LangChain4j tutorial. This application is enhanced with the GraalVM Native Image Maven plugin that enables you to easily create a native executable from a Java application.

The goal of this example is to demonstrate:

  • how easy it is to create a CLI application (that uses LangChain) using ahead-of-time (AOT) compilation
  • how to optimize a native executable for file size

Prerequisites

Note: To interact with OpenAI, store your OpenAI API key as the value of an environment variable named OPENAI_API_KEY.

Package the Application as a Native Executable with Maven

  1. Clone this repository with Git and then enter it:

    $ git clone https://github.com/olyagpl/langchain4j-searchbot.git
    $ cd langchain4j-searchbot

  2. Compile and create a JAR file with all dependencies (required for Native Image):

    $ mvn clean package

  3. Create a native executable using the Maven plugin for Native Image:

    $ mvn -Pnative package

    This produces a Linux self-contained executable version of the searchbot!

  4. Interact with the searchbot:

    $ ./target/searchbot “Why should I visit Lviv?”

Create a Native Executable Optimized for File Size

You can optimize your native executable by taking advantage of different optimization levels. The following levels are available:

# native-image --help output
    -O                    control code optimizations: b - optimize for fastest build time, s
                          - optimize for size, 0 - no optimizations, 1 - basic
                          optimizations, 2 - advanced optimizations, 3 - all optimizations
                          for best performance.

For this application, use the -Os option to create the smallest possible native executable.

  1. Create a native executable with the file size optimization on, and giving it a different name:

    $ native-image \
    -Os \
    -jar ./target/searchbot-1.0-jar-with-dependencies.jar \
    -H:+AllowDeprecatedBuilderClassesOnImageClasspath \
    -o ./target/searchbot-optimized

  2. Interact with the searchbot to test it:

    $ ./target/searchbot-optimized “What is JavaDay Lviv?”

  3. Compare the sizes of all relevant output files:

    $ du -h target/searchbot*

    You should see the output similar to this:

    35M	target/searchbot
208M	target/searchbot-1.0-jar-with-dependencies.jar
8.0K	target/searchbot-1.0.jar
31M	target/searchbot-optimized

There are other Native Image techniques that can improve your application performance and positively affect the executable size, for example Profile-Guided Optimizations (PGO). The improvement degree depends on the application complexity. Find more in the Native Image Optimizations and Performance documentation.

Learn More