Ruby LangChain Testing Guide 2025 | RSpec + WebMock

Tests that call a real model are slow, cost money and give a different answer each time. For langchainrb code the fix is to stop the request at the HTTP layer with WebMock and let the gem run its real code against a canned reply. That tests the part you own: how you build messages, read the reply, and handle failures.

Everything below ran on Ruby 4.0.6 with langchainrb 0.19.5 (the latest release on RubyGems, published 2025-05-01), ruby-openai 8.3.0, RSpec 3.13 and WebMock 3.34. No real API key and no network were used. I did not run VCR end to end, and I did not write a CI workflow, so this post has neither.

Why stub HTTP and not the LLM object #

Two common ways to fake a model:

• Stub the LLM object (allow(llm).to receive(:chat)). Fast, but your test now checks your stub. The gem’s request building and response parsing never run.
• Stub the HTTP request with WebMock. The real Langchain::LLM::OpenAI runs, builds a real request, parses a real-shaped response. Your assertion can check what was sent as well as what came back.

Use the object stub for tests of your own classes that merely call llm.chat. Use WebMock where you want to know the gem is wired the way you think.

The spec #

# spec/spec_helper.rb
require "langchain"
require "webmock/rspec"
WebMock.disable_net_connect!

RSpec.describe Langchain::LLM::OpenAI do
  subject(:llm) { described_class.new(api_key: "test-key") }

  def stub_chat(status: 200, body: nil)
    body ||= { id: "c1", model: "gpt-test",
               choices: [{ index: 0, message: { role: "assistant", content: "Hello from RSpec" }, finish_reason: "stop" }],
               usage: { prompt_tokens: 8, completion_tokens: 3, total_tokens: 11 } }
    stub_request(:post, "https://api.openai.com/v1/chat/completions")
      .with(headers: { "Authorization" => "Bearer test-key" })
      .to_return(status:, body: body.to_json, headers: { "Content-Type" => "application/json" })
  end

  it "chat returns a response object with text and token counts" do
    stub = stub_chat
    response = llm.chat(messages: [{ role: "user", content: "Say hello" }])
    expect(response).to be_a(Langchain::LLM::OpenAIResponse)
    expect(response.chat_completion).to eq("Hello from RSpec")
    expect([response.prompt_tokens, response.completion_tokens, response.total_tokens]).to eq([8, 3, 11])
    expect(stub).to have_been_requested.once
  end

  it "sends the messages in the request body" do
    stub_chat
    llm.chat(messages: [{ role: "user", content: "Say hello" }])
    expect(a_request(:post, "https://api.openai.com/v1/chat/completions")
      .with { |req| JSON.parse(req.body)["messages"] == [{ "role" => "user", "content" => "Say hello" }] })
      .to have_been_made
  end

  it "blocks unstubbed requests" do
    expect { llm.chat(messages: [{ role: "user", content: "x" }]) }
      .to raise_error(WebMock::NetConnectNotAllowedError)
  end
end

My full run had 9 examples and 0 failures: these three, plus complete, embed, the three failure stubs below, and the Assistant spec. The last test above is a safety net. With disable_net_connect!, a code path you forgot to stub fails loudly instead of quietly calling OpenAI.

Two facts the examples confirm about the gem, which the earlier version of this post got wrong:

• chat returns a Langchain::LLM::OpenAIResponse, not a string. The text is response.chat_completion, and token counts are prompt_tokens, completion_tokens and total_tokens. complete(prompt:) also returns a response object, read with response.completion.
• Calling chat without a model sent "model":"gpt-4o-mini" in the request body in 0.19.5, so the gem picks a default for you. Pass the model explicitly if the choice matters to your bill.

What your code has to rescue #

The earlier version of this post rescued Langchain::LLM::ApiError. That class does not exist in 0.19.5: Langchain::Errors contains only BaseError. The gem passes HTTP failures through from Faraday. I stubbed three failures and printed what was raised:

So rescue Faraday::TooManyRequestsError is the honest way to back off on rate limits, and rescue Faraday::Error is the broad net. The timeout row is how WebMock’s to_timeout surfaced here. A real socket timeout may raise a different Faraday class, so test the exact failure you care about against the real client before relying on this table.

Conversation state: use Langchain::Assistant #

An older version of this guide described memory classes (buffer, summary, entity). I checked: in 0.19.5 there is no Langchain::Memory constant. The gem’s conversation holder is Langchain::Assistant, which keeps messages and resends them on each turn.

RSpec.describe Langchain::Assistant do
  let(:llm) { Langchain::LLM::OpenAI.new(api_key: "test-key") }
  # stub_request(:post, ".../chat/completions") returns "Hi Ann", 20 prompt + 2 completion tokens

  it "keeps the conversation and resends it on the next turn" do
    assistant = described_class.new(llm:, instructions: "Be brief.")
    assistant.add_message_and_run!(content: "I am Ann")
    assistant.add_message_and_run!(content: "Who am I?")

    expect(assistant.messages.map(&:role)).to eq(%w[system user assistant user assistant])
    # request 2 carried: ["Be brief.", "I am Ann", "Hi Ann", "Who am I?"]
  end
end

It passed. The second request’s body contained all four texts, which is what you want to assert: history is resent, so every turn costs more than the last (see token costs). Note that in 0.19.5 each message’s content is sent as a list of {"type": "text", "text": ...} parts, not a bare string, so a request-body assertion must dig into content[0]["text"]. After two turns, total_prompt_tokens was 40, total_completion_tokens 4 and total_tokens 44, the sums of my two stubbed usage blocks. That is a cheap way to unit-test any per-user token limit you add.

Logging: check before you ship #

In Langchain::LLM::OpenAI#initialize (0.19.5) the gem installs a Faraday logger with headers: true, bodies: true on Langchain.logger. The README says the logger defaults to Logger::DEBUG. In my test run, the log lines at INFO level printed each request, including the header Authorization: "Bearer test-key", and each request and response body.

That means in an app with default settings, your API key and your users’ prompts go to STDOUT. Set the level yourself, in your app config and in the test helper:

Langchain.logger.level = Logger::WARN

I verified the default level (0, DEBUG) and the log output. I did not test which level suppresses which lines, so check your own output after setting it.

VCR, if you want recorded replies #

VCR records real HTTP interactions once and replays them. It is useful when you want a reply shaped exactly like production. I confirmed the configuration methods exist in VCR 6.4.1 (filter_sensitive_data, default_cassette_options, hook_into, record modes :all, :none, :new_episodes, :once) but did not run a recording. If you use it, scrub the key before the cassette hits disk:

VCR.configure do |c|
  c.hook_into :webmock
  c.cassette_library_dir = "spec/cassettes"
  c.filter_sensitive_data("<OPENAI_API_KEY>") { ENV["OPENAI_API_KEY"] }
  c.default_cassette_options = { record: :none } # in CI, replay only
end

A recorded reply is a photograph of one old answer. It tests your parsing; it does not tell you the model is still good. For that, see the eval approach in testing and monitoring LLM apps.

What I did not verify #

• Real OpenAI calls, rate-limit headers, streaming, tool calls.
• Other providers in the gem (Anthropic, Gemini, Ollama) and their error classes.
• A CI workflow, coverage thresholds, or performance tests. I have no measurements to offer.

Sources #

• langchainrb README - Langchain::LLM::OpenAI, chat, complete, logger defaulting to DEBUG
• langchainrb on RubyGems - release history; 0.19.5 is the latest as of 2026-10-30
• WebMock - stub_request, disable_net_connect!, to_timeout
• VCR - cassettes, record modes, filter_sensitive_data
• Faraday - the error classes the HTTP layer raises

Related: getting started with langchainrb and RubyLLM vs langchainrb.