Adding streaming endpoints

Overview

In this step of the guide, you’ll add some endpoints to provide a client-friendly API in front of all of the RAG components you’ve been building. You’ll create an API for submitting your "Ask Akka" questions (prompts), and one that serves up a self-hosted, static asset web UI.

Prerequisites

You will need to have your MongoDB Atlas database URL and your Open AI API key available, as they are required to run the Ask Akka service.

If you are following along with each step rather than using the completed solution, then you’ll need the code you wrote in the previous step.

Adding a streaming API

Akka HTTP endpoints have the ability to return streams of data via server-sent events (SSE). For more information on this feature, check out the Akka SSE documentation.

AskHttpEndpoint.java
@Acl(allow = @Acl.Matcher(principal = Acl.Principal.INTERNET))
@HttpEndpoint("/api")
public class AskHttpEndpoint {

  public record QueryRequest(String userId, String sessionId, String question) {
  }

  private final ComponentClient componentClient;
  private final AskAkkaAgent askAkkaAgent; (1)
  private final Materializer materializer;

  public AskHttpEndpoint(AskAkkaAgent askAkkaAgent, Materializer materializer, ComponentClient componentClient) {
    this.askAkkaAgent = askAkkaAgent;
    this.materializer = materializer;
    this.componentClient = componentClient;
  }

  /**
   * This method runs the search and streams the response to the UI.
   */
  @Post("/ask")
  public HttpResponse ask(QueryRequest request) {

    var response = askAkkaAgent
        .ask(request.userId, request.sessionId, request.question)
        .map(StreamedResponse::content); (2)

    return HttpResponses.serverSentEvents(response); (3)
  }
}
1 Inject the AskAkkaAgent class created in the previous step
2 Extract the content field from each StreamedResponse
3 Use the serverSentEvents function to easily send a stream via SSE

The userId and sessionId parameters are required in QueryRequest along with the question field.

Adding the users API

There is a convenience endpoint that you can use to query the list of sessions for a given user:

UsersEndpoint.java
@Acl(allow = @Acl.Matcher(principal = Acl.Principal.INTERNET))
@HttpEndpoint("/api")
public class UsersEndpoint {

  private final ComponentClient componentClient;

  public UsersEndpoint(ComponentClient componentClient) {
    this.componentClient = componentClient;
  }

  @Get("/users/{userId}/sessions/")
  public ConversationHistoryView.ConversationHistory getSession(String userId) {

    return componentClient.forView()
        .method(ConversationHistoryView::getSessionsByUser)
        .invoke(userId);
  }
}

One subtle thing worth pointing out here is that both the streaming RAG endpoint and the user view query endpoint have the exact same route as defined in @HttpEndpoint("/api").

Adding the static UI endpoint

You can now add an endpoint that serves up the static UI. This is surprisingly simple in Akka, as the HTTP endpoint class has built-in support for serving these kinds of assets.

UiEndpoint.java
@HttpEndpoint
@Acl(allow = @Acl.Matcher(principal = Acl.Principal.ALL))
public class UiEndpoint {
  @Get("/")
  public HttpResponse index() {
    return HttpResponses.staticResource("index.html"); (1)
  }
}
1 The staticResource function serves up a file from main/resources/static-resources

And lastly, we just need to fill out the index.html file to provide the static UI for Ask Akka.

There is far too much code in the HTML file to list out here. If you want to run the UI with the Ask Akka service, here you might want to switch to the version that is in the repository so you can get all of the single-file React code.

Running the service

Running the service should now just be a matter of running mvn compile exec:java. Make sure that you have set both the OPENAI_API_KEY and MONGODB_ATLAS_URI environment variables before running exec:java.

If you haven’t run the indexer yet, do so with:

curl -XPOST localhost:9000/api/index/start

Once you’ve made sure that your MongoDB Atlas database has a functioning and properly named vector index, you can use the Ask Akka service with a simple curl command:

curl localhost:9000/api/ask --header "Content-Type: application/json" -XPOST \
--data '{ "userId": "001", "sessionId": "foo", "question":"How many components exist in the Akka SDK?"}'

Next steps

Now that you’ve gone through the process of building the Ask Akka sample, you should start playing with it and even breaking it. Change the indexing parameters like chunk size and see if that affects how the LLM performs. The key is to roll up your sleeves and get dirty, as that’s the best way to extend your learning beyond what’s covered in this guide.

Make sure you check out our thorough discussion of agentic AI and where Akka fits in the ecosystem.