How to pilot a conversation service before building software

A conversation service can look straightforward in a product sketch: a person asks a question, an operator responds, and the request reaches a useful outcome. The work between those steps is often where the real design decisions live. Who is available to answer? Which questions require somebody else's permission? What happens when the information is incomplete or the person leaves before the issue is resolved?

A small manual pilot can help a founder observe that work before committing to custom software. The purpose is to learn whether a defined audience wants a defined service and what delivering it requires. The example plan below is illustrative. It is not a claim that a particular business model has been tested or that a manual trial predicts commercial success.

Choose one audience and one request

Start with a narrow sentence. A pilot might help a small group of independent merchants answer product-care questions during one afternoon period. Another might help guests at one property understand arrival instructions. These are different pilots because they involve different information, responsibilities, and routes to the user.

Avoid beginning with a general promise to answer anything. That makes it difficult to distinguish demand for the intended service from requests the future business would never accept. Write down the included request types and a few examples of requests that must go elsewhere. The person running the pilot needs those boundaries as much as the participant does.

The GOV.UK guidance on learning user needs emphasizes understanding what people are trying to achieve and the problems in their current situation. Apply that idea by asking about a recent request, rather than asking whether somebody likes the concept of a new chat product. A concrete account of what happened provides material that a pilot can test.

Write the question the pilot should answer

A useful pilot question might be whether merchants will delegate product-care questions when the service uses an approved knowledge pack. That question points toward observable behavior: a merchant provides the pack, allows a limited trial, and reviews the answers. It also identifies the operational work that needs to be observed.

Separate that question from other uncertainties. A manual pilot may teach a founder about staffing and handoffs while revealing very little about a future subscription price. A small invited group may provide useful feedback but tell little about acquisition through paid advertising. Record those limits at the start so that the results are not asked to support a larger conclusion than the experiment can bear.

Choose a few measures that relate directly to the question. Examples include requests within scope, requests needing escalation, time spent finding approved information, and cases still unresolved when the coverage period ends. Message count alone may obscure the work: a short exchange can require a difficult decision, while a longer one may involve routine clarification.

Prepare the smallest complete service

The pilot needs an invitation, a way to submit a request, somebody responsible for responding, and a method for recording the outcome. It also needs an explanation of service hours and a fallback when the pilot cannot help. These elements make a complete experience even if they are supported by existing tools and manual work.

Choose tools the team can operate safely and consistently. A shared inbox or an existing support system may be enough for a limited trial. Do not ask participants to place sensitive information in an informal document simply because it is convenient. Decide who can access the records, which details are necessary, and how records will be handled after the trial.

The GOV.UK guide to making prototypes advises choosing the kind of prototype that matches what needs to be tested. A conversation-service pilot may need a realistic invitation and response process more than a polished interface. If the research question concerns a complex interaction, a more functional prototype may be appropriate. Match the effort to the uncertainty.

Rehearse the difficult cases

Before inviting participants, run a few fictional requests through the process. Include an ordinary question, a request outside scope, a missing piece of information, and an issue that needs approval from another person. Ask the operator to show where the record lives and how the next person knows what to do.

A rehearsal can expose practical gaps without involving a real customer. Perhaps the knowledge pack has no owner, or the person who can approve an exception is unavailable during the proposed hours. Resolve those issues before advertising coverage. Otherwise, the pilot may measure a preventable setup failure instead of the service idea.

Also test the end of a shift. An unfinished request needs a clear owner after the scheduled coverage stops. The participant should receive an accurate explanation of the next step. The team needs to know whether the pilot operator continues to follow the request or hands it to an established channel.

Run a bounded two-week trial

A proposed two-week schedule can begin with a few days of preparation, followed by a limited set of staffed sessions and a final review. The exact duration should fit the audience and request frequency. A business with infrequent relevant requests may need a different observation period. A calendar date is a boundary for the experiment, not evidence that enough learning has occurred.

During the staffed sessions, keep a simple decision log. Record the request type, the source used to answer it, any dependency on another person, and the outcome. Capture the time spent on preparation and follow-up as well as the visible conversation. That helps the founder see work that a future interface might otherwise hide.

Ask participants for feedback close to the interaction. What were they trying to do? Did they understand the answer and the next step? Where did they expect something different? Avoid asking only whether they were satisfied. A polite positive response can coexist with a misunderstanding about what the service actually did.

Review the economics without inventing scale

A manual trial provides observations about this trial. It does not establish that costs will fall automatically as volume grows. Review the work that repeats, the work requiring judgment, and the work caused by missing information. Some tasks may be suitable for automation; others may require a better agreement with the customer or a narrower service scope.

Separate one-time setup effort from recurring delivery work, while keeping both visible. Building a knowledge pack for a merchant is different from checking an answer during a shift. A service may still need to recover both costs. Before choosing a pricing model, understand which activities change with each customer, each session, and each request.

Decide what happens next

Write the stop and continuation conditions before reviewing the results. A founder might decide to continue only if participants use the service for the intended request, the team can explain the workload, and unresolved cases have an acceptable route. If the trial fails those conditions, the next step may be a narrower offer or a different audience rather than a larger software build.

End with a one-page record: the question tested, who participated, what happened, what remains uncertain, and the next decision. Include awkward results and incomplete cases. A useful pilot reduces a specific uncertainty and makes the next investment easier to judge. The first action is therefore modest: draft that one-page plan, recruit a relevant participant group, and rehearse the service before the first real request arrives.