How to start using SambaNova Cloud for AI inference

Start using SambaNova Cloud by getting a SambaCloud account and API key, choosing a currently available model, and sending a small test through its OpenAI-compatible API. Inference means asking a trained model to produce an answer. Keep the tested prompt and findings in fenbs so you can continue the experiment in another AI without repeating the early work.

SambaNova’s current product wording is SambaCloud. Its developer overview↗ (opens in a new tab) covers both hosted SambaCloud and SambaStack deployments. For a hosted first test, follow the SambaCloud instructions. SambaStack users need their administrator’s local endpoint and key-generation details instead.

What do you need before the first request?

You need access to SambaCloud, a valid key, and the exact model ID you intend to call. A model ID is the name the API uses to select a model. Copy a currently available ID rather than treating an example from an old tutorial as permanent.

Follow SambaNova’s API keys and URLs instructions↗ (opens in a new tab) to obtain and securely save the key. The vendor says keys cannot be viewed again after creation. Keep it in an approved secret store or a local environment variable, not in a prompt, shared notebook, source file, or saved test summary.

1. Confirm that you are testing hosted SambaCloud, rather than a local SambaStack deployment.
2. Obtain the API key and save it securely before leaving its creation screen.
3. Copy the exact current model ID for your test.
4. Prepare a short prompt whose answer you can check yourself.
5. Write the expected answer before making the request.

For example, ask the model to extract three task names from a harmless paragraph and return only those names. This makes it possible to spot an omitted item, an invented task, or unwanted commentary. Start with a test you understand before using a long document whose mistakes are harder to find.

What should you try in a browser playground?

If you already have playground access in your account, use it for a first look at the same small prompt. The developer overview linked above does not give a click-by-click playground menu path. Use the options actually available in your signed-in account rather than looking for an assumed button name.

Choose an available model and send the test. Keep the prompt unchanged while you inspect whether the answer follows the requested format and contains only the expected facts. Write down the model ID and any settings you deliberately changed so the result has an understandable starting point.

Do not infer that a browser result proves your application is configured correctly. The API test below checks that your code uses the intended endpoint, key, and model. If playground access is unavailable, the documented API route still provides a concrete way to make your first hosted-model request.

How do you call the OpenAI-compatible API?

For OpenAI-compatible requests, SambaCloud uses the base URL https://api.sambanova.ai/v1. Compatibility means you can use the OpenAI client’s request shape while pointing it at SambaNova. It does not mean the request is sent to OpenAI or that every provider-specific feature works unchanged.

The example below is a minimal Python client test. Make the OpenAI Python package available in your development environment, then set SAMBANOVA_API_KEY and SAMBANOVA_MODEL outside the source file. Use your current model ID for the second value; the placeholder deliberately avoids freezing an older model into the example.

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ['SAMBANOVA_API_KEY'],
    base_url='https://api.sambanova.ai/v1',
)

response = client.chat.completions.create(
    model=os.environ['SAMBANOVA_MODEL'],
    messages=[{
        'role': 'user',
        'content': (
            'Return only the task names, one per line: '
            'Maya will draft the outline. Lee will review the draft. '
            'Sam will check the links.'
        ),
    }],
)

print(response.choices[0].message.content)

Run the test in an environment where you can inspect the output. Compare it with the three tasks in the source paragraph. A request that returns an answer has passed a connection check; the content still needs its own review. The AI API key setup guide covers the broader handling of credentials.

How do you check that the test means anything?

Separate the connection result from the task result. First confirm that the request reached the intended provider with the intended model. Then check whether the response meets your acceptance rules. Those questions need different fixes when something goes wrong.

Repeat the prompt with one deliberate change, such as an extra sentence that contains no task. Check whether the model wrongly includes it. This is an example of testing your own use case, not a benchmark claim about the model’s overall ability.

Keep those observed limits through fenbs. A later assistant can use the same test and distinguish a verified finding from an assumption.

What should you check when a request fails?

Inspect the error before changing the prompt. Confirm that the environment variable is present, that you copied the current model ID exactly, and that this OpenAI-compatible client uses the base ending in /v1. Keep credentials out of anything you share while debugging.

Do not mix setup instructions for different compatibility routes. SambaNova documents a separate Anthropic-compatible configuration for Claude Code↗ (opens in a new tab), whose base URL does not include /v1. Its direct mode supports Claude Code’s configured MCP servers. That is a different client setup, so copying its values into this example would change the configuration you are testing.

Connecting a model also does not automatically connect outside tools. SambaNova’s FastMCP guide↗ (opens in a new tab) describes client code that connects tools to a suitable model. MCP is a standard way for an AI app to use an outside tool; the hosted inference endpoint does not discover arbitrary tool servers by itself.

For an editor-based route, see using SambaNova models in Cline. For another provider’s first-request procedure, use the separate Gemini API getting-started guide. Keep each provider’s setup and findings distinct.

How to keep your inference findings with fenbs

Keep the work so you can carry on from exactly where you left off, on any device and in another AI. Save the tested prompt, model ID, observed result, and next check. Leave the API key in its approved secret store.

1. Create a free account.
2. Write what you tested, which model you used, and what a good answer should contain.
3. Save the harmless prompt and your findings, including anything that failed.
4. Connect SambaNova to fenbs through Claude Code using the connection guides. Choose the direct mode described in SambaNova’s guide above.
5. Ask the connected AI to read your findings before the next experiment and save where to continue when you stop.

fenbs does not host SambaNova models or run the inference request for you. For a one-off answer you will not revisit, saving a separate work record may be unnecessary. Use fenbs when the experiment needs to continue across sessions or AI apps. The connect your AI guides and OpenAI API integration guide offer further reading on separating model access from the rest of an application.