Real time search is a common feature that is included in products. A typeahead is a common example of that. The idea is that as the user types, search results return in real time. Slack does this when you first start to mention a user. This post looks at how to build a robust real time search feature by avoiding some common mistakes.
The goal of this post is to create a search bar that searches the names of the Font Awesome icons and displays the icons and names that match the search term. We will create the front end using React and RxJS. We’ll look at a naive approach at first and discuss the pitfalls with the implementation. Then we’ll look at a robust implementation using RxJS. The final version is demoed here. See the screenshot below for the desired result:
The HTTP API Powering the Real Time Search
I created an example API for this post to search the Font Awesome icons. It searches for different icons in the free Font Awesome icon set. The API returns a list of objects containing uri for the svg file and the name of the icon. There is an artificial delay added in the API to simulate a real API. The more search results there are, the longer it will take to respond. This will be explained later in the article. The API is deployed here. The source code of the API can be found on GitHub. Here’s an example request:
GET https://fa-search-backend.herokuapp.com/search?term=user-circleAnd here’s the response:
[
{"uri":"svgs/regular/user-circle.svg","name":"user-circle"},
{"uri":"svgs/solid/user-circle.svg","name":"user-circle"}
]The SVG icons are hosted on the API as well and can be accessed by prepending the API URL to the relative URI given in the response. Like this.
Step 1 – Set Up UI
The GitHub repository starts from a create React app base. Then there are some changes.
There’s an input component for typing in the search term:
// Input.js
import React from 'react';
function Input({loading, onChange}) {
return (
<div className="input-group mb-3 mt-3">
<input type="text" className="form-control" placeholder="Search..." aria-label="Search icons" aria-describedby="basic-addon2" onChange={onChange} />
<div className="input-group-append">
{ loading ?
<span className="input-group-text" id="basic-addon2">Loading...</span> :
<span className="input-group-text" id="basic-addon2">Ready</span>
}
</div>
</div>
);
}
export default Input;There’s a search results component to show the results.
// Results.js
import React from 'react';
function Results({data = [], errorMessage = '', noResults = false}) {
return (
<div>
{ errorMessage && <div> {errorMessage} </div> }
{ noResults && <div> No results for this search </div> }
<div className="grid">
{ data.map(({name, uri}) =>
<div key={uri} className="card grid-child">
<img src={uri} className="card-img-top" alt={name} />
<p className="card-text"><a href={'https://fontawesome.com/icons/' + name}>{name}</a></p>
</div>
)}
</div>
</div>
);
}
export default Results;
Then there is a container component, containing all the logic. It will start off looking like this:
// Container.js
import React from 'react';
import Input from './Input.js';
import Results from './Results.js';
function Container() {
return (
<div className="container">
<Input />
<Results />
</div>
);
}
export default Container;In the index.js file, the App component is replaced with the Container component. App.js, App.css, and App.test.js can be deleted. The index.css file is replaced with these contents:
.grid {
display: grid;
grid-gap: 10px;
grid-template-columns: repeat(auto-fit, 100px);
}
.grid-child {
height: 200px;
}
.card-img-top {
height: 100px;
}The result should look something looking like this:
Here’s all the code for step 1.
Step 2 – A Naive Approach
We’ll start off with a very naive implementation of this. We’ll hit the API on every keypress. At the end of this step, we’ll discuss some of the issues with that approach.
First, the container will hold the state for the results. We’ll import the state hook from React and we’ll add an object in state called state. It will hold the data, error message, and if the data if loading.
const [state, setState] = useState({
data: [],
loading: false,
errorMessage: ''
});Then we’ll pass that on to the Input and Results components:
<Input loading={state.loading} />
<Results data={state.data} errorMessage={state.errorMessage} />Next, we’ll create an onChange handler to get the text from the input, call the API, and update the state.
const onChange = e => {
const term = e.target.value;
// At the start of each request set the loading state to true and clear the data
setState(s => ({
loading: true,
data: [],
errorMessage: ''
}));
//Use browser fetch API to search
fetch('https://fa-search-backend.herokuapp.com/search?delay=true&term=' + term).then(response => {
return response.json();
}).then(data => {
//Set the response data
setState(s => ({
loading: false,
data,
errorMessage: ''
}));
}).catch(e => {
// Set the error state.
setState(s => ({
loading: false,
data: [],
errorMessage: e.message
}));
});
};
Then we need to pass this handler to the Input component:
<Input loading={state.loading} onChange={onChange} />The Results component (in Results.js) needs to be updated to have the API prefixed before the image on line 11:
<img src={'https://fa-search-backend.herokuapp.com/' + uri} className="card-img-top" alt={name} />Okay – now we have a working typeahead! Kind of. The video below shows some of the bugs.
The API simulates a real-life API. In a real-world API when you’re doing a search the first few characters are going to take the longest to search. The later requests may respond before the initial requests. That means the responses to the earlier requests overwrite the responses to the later requests showing the wrong results for the search term. Also, firing a request off on each change means that the server may be overloaded with requests that aren’t even needed by the user. Another issue is that requests with zero to one character will take longer and the user may not need the responses from those.
Step 3 – Set up RxJS
One tool that handles this type of data very well is RxJS. It can be used to combine different asynchronous streams of data to get the outcome we desire. Let’s install it via npm npm install rxjs (or yarn if you use that, yarn add rxjs).
Let’s set up the Container component to do the same thing it’s doing now, but with RxJS, then we’ll look at how to solve some of the problems listed above with the new set up. We’ll get the data from the input as a stream using a BehaviorSubject. Then we’ll subscribe to it to get the changes. We’ll also put the subject in a state hook since the Input component depends on it. We want to initialize the subject when the component loads and subscribe to changes. We’ll use the effect hook to set that up. The hook will only be called when the subject changes. We’ll only initialize a subject if it is null.
const [subject, setSubject] = useState(null);
useEffect(() => {
if(subject === null) {
const sub = new BehaviorSubject('');
setSubject(sub);
} else {
// Since this effect re-runs when subject is changed,
// after it is set we'll subscribe to the changes
subject.subscribe( term => {
return fetch('https://fa-search-backend.herokuapp.com/search?delay=true&term=' + term).then(response => {
return response.json();
}).then(data => {
const newState = {
data,
loading: false
};
setState(s => Object.assign({}, s, newState));
});
});
// When the component unmounts, this will clean up the
// subscription
return () => subject.unsubscribe();
}
}, [subject]);We’ll update the onChange handler to pass the input value to subject.next if it’s set:
const onChange = e => {
if(subject) {
return subject.next(e.target.value);
}
};Now this does the same thing as the previous code, but with RxJS. Now the Container component is in a position to leverage the features of RxJS to solve the problems discovered in the naive approach.
Step 4 – Utilize RxJS
Here’s a number of problems with the naive approach:
- Make a request on every keystroke – can overwhelm API – slows page down with many renders
- Duplicate requests / requests with spaces
- Making requests with very little input
- Results rendered out of order
The above problems lead to a poor user experience in an application. They will be addressed one by one below.
Problem 1 – Too many requests
The first problem can be solved with the debouceTime operator of RxJS. This only sends an event if there’s been a break between events of a specified time. For reference, see here and here. Below is the update code:
useEffect(() => {
if(subject === null) {
const sub = new BehaviorSubject('');
setSubject(sub);
} else {
const observable = subject.pipe(
debounceTime(200)
).subscribe( term => {
return fetch('https://fa-search-backend.herokuapp.com/search?delay=true&term=' + term).then(response => {
return response.json();
}).then(data => {
const newState = {
data,
loading: false
};
setState(s => Object.assign({}, s, newState));
});
});
return () => {
observable.unsubscribe()
subject.unsubscribe();
}
}
}, [subject]);
There’s a pipe added on the end of the subject and the operator is placed inside the pipe. That solves the problem of not sending a request, every request. It also seems to solve the problem of the out of order requests. However, it doesn’t. It covers it up by delaying sending earlier requests until the input stops changing for 200 ms. Once there’s a break of 200 milliseconds between input changes, it emits the value of the input. If you would type us, wait 200 ms, then type user circle, the issue of rendering the requests out of order will be observed. We’ll address that later.
Problem 2 – Duplicate requests
There is also an issue of sending the same search term muliple times or sending it with a space (ex user vs user). RxJS has the map and the distinctUntilChanged operators to solve these issues. map can be used to trim the string and the distinctUntilChanged operator will only emit an event if the current value in the stream is different than the previous value. Below is the updated code. The operators are added to the pipe.
const observable = subject.pipe(
map(s => s.trim()),
distinctUntilChanged(),
debounceTime(200)
).subscribe( term => {
return fetch('https://fa-search-backend.herokuapp.com/search?delay=true&term=' + term).then(response => {
return response.json();
}).then(data => {
const newState = {
data,
loading: false
};
setState(s => Object.assign({}, s, newState));
});
});Problem 3 – Requests with little input
Another issue with the naive approach is that the application is sending API requests with very little input that take a lot of time. The first event is a blank string and it will send only one letter if there are no other ones added. To solve this next issue, the application will only send a request if there are two or more characters. It will use the filter operator from RxJS. The code is updated below:
const observable = subject.pipe(
map(s => s.trim()),
distinctUntilChanged(),
filter(s => s.length >= 2),
debounceTime(200)
).subscribe( term => {
return fetch('https://fa-search-backend.herokuapp.com/search?delay=true&term=' + term).then(response => {
return response.json();
}).then(data => {
const newState = {
data,
loading: false
};
setState(s => Object.assign({}, s, newState));
});
});Problem 4 – out of order requests
One of the biggest issues with the naive approach is out of order responses. The responses of the earlier requests will overwrite the responses of the later requests. The operator that will solve this issue is the switchMap operator. The documentation on learnrxjs is a little more clear. This will only take the response of the latest requests and ignore ones that are late. This will require us to move the fetch statement out of the subscribe and into the pipe operators. RxJS will automatically convert the promise into an observable. Below is the updated code:
const observable = subject.pipe(
map(s => s.trim()),
distinctUntilChanged(),
filter(s => s.length >= 2),
debounceTime(200),
switchMap(term => {
return fetch('https://fa-search-backend.herokuapp.com/search?delay=true&term=' + term).then(response => {
return response.json();
}).then(data => {
return {
data,
loading: false
};
});
})
).subscribe( newState => {
setState(s => Object.assign({}, s, newState));
});Now the real time search is much more robust, rendering the results in order of the requests and discarding unused requests.
Step 5 – Polish and clean up
Despite the real time search working better, there’s still a few issues. The application only goes the happy path. Any error from the server will break the application. The loading indicator was taken away. If there are no results, it should be made more clear to the user. These issues are summarized below:
Loading Indicator
While the application is getting data from the server, we’ll want to indicate to the user that the data is loading. This can be achieved using the merge operator and the creation operator of. We’ll merge of({loading: true}) with the fetch function. The loading event will get emitted first, then when fetch promise resolves it’ll emit the data. If you noticed, the new state is merged with the existing state. This is so that the existing state.data does not get set to an empty array when the loading event is emitted. See the below code for an update:
const observable = subject.pipe(
map(s => s.trim()),
distinctUntilChanged(),
filter(s => s.length >= 2),
debounceTime(200),
switchMap(term =>
merge(
of({loading: true}),
fetch('https://fa-search-backend.herokuapp.com/search?delay=true&term=' + term).then(response => {
return response.json();
}).then(data => {
return {
data,
loading: false
};
})
)
)
).subscribe( newState => {
setState(s => Object.assign({}, s, newState));
});Error Handling
The application crashes if the server returns an error. The API is set up so that if the term error is passed it’ll return a 500 error. The server error needs to be handled in the fetch response. Application errors also need to be handled with the RxJS operator catchError. The error message also needs to be reset at the beginning of each request. The code update below handles those:
const observable = subject.pipe(
map(s => s.trim()),
distinctUntilChanged(),
filter(s => s.length >= 2),
debounceTime(200),
switchMap(term =>
merge(
of({loading: true, errorMessage: ''}),
fetch('https://fa-search-backend.herokuapp.com/search?delay=true&term=' + term).then(response => {
if(response.ok) {
return response
.json()
.then(data => ({data, loading: false}));
}
return response
.json()
.then(data => ({
data: [],
loading: false,
errorMessage: data.title
}));
})
)
),
catchError(e => ({
loading: false,
errorMessage: 'An application error occured',
data: []
}))
).subscribe( newState => {
setState(s => Object.assign({}, s, newState));
});No results indication
If there are no results the page is blank, just like at the beginning of a search. This isn’t very helpful to a user; they may be confused and think the page is broken. It would be helpful to indicate to them that there are no results. One way would be to check if state.data is empty. However, that’s not a good indication because it may be that way if the user hasn’t started their search. What we’re going to do here is introduce a noResults property on the state. At the beginning of the request, it will be set to false and at the end of the request it’ll be set to state.data.length === 0. In the Results component, if that property is set to true, then the application will display a message to the user saying that no search results were found matching the term.
Here’s the updated Container component code. The Results component already handles the noResults property.
import React, { useState, useEffect } from 'react';
import { BehaviorSubject, of, merge } from 'rxjs';
import { debounceTime, map, distinctUntilChanged, filter, switchMap, catchError } from 'rxjs/operators';
import Input from './Input.js';
import Results from './Results.js';
function Container() {
const [state, setState] = useState({
data: [],
loading: false,
errorMessage: '',
noResults: false
});
const [subject, setSubject] = useState(null);
useEffect(() => {
if(subject === null) {
const sub = new BehaviorSubject('');
setSubject(sub);
} else {
const observable = subject.pipe(
map(s => s.trim()),
distinctUntilChanged(),
filter(s => s.length >= 2),
debounceTime(200),
switchMap(term =>
merge(
of({loading: true, errorMessage: '', noResults: false}),
fetch('https://fa-search-backend.herokuapp.com/search?delay=true&term=' + term).then(response => {
if(response.ok) {
return response
.json()
.then(data => ({data, loading: false, noResults: data.length === 0}));
}
return response
.json()
.then(data => ({
data: [],
loading: false,
errorMessage: data.title
}));
})
)
),
catchError(e => ({
loading: false,
errorMessage: 'An application error occured'
}))
).subscribe( newState => {
setState(s => Object.assign({}, s, newState));
});
return () => {
observable.unsubscribe()
subject.unsubscribe();
}
}
}, [subject]);
const onChange = e => {
if(subject) {
return subject.next(e.target.value);
}
};
return (
<div className="container">
<Input loading={state.loading} onChange={onChange} />
<Results data={state.data} errorMessage={state.errorMessage} noResults={state.noResults} />
</div>
);
}
export default Container;Now the search is pretty robust. It shows the user when there are no results, it handles errors, and works as expected. Here’s a video demonstrating everything we addressed.
You can interact with the final result here.
Real Time Search Conclusion
Another feature that could be added to this search page is adding the search term to the url. The application should load the results using the term in the url when the page initially loads. Another feature is clearing the results when the input is completely erased.
There is a lot that goes into making a robust real time search. What may seem like a simple problem is actually more complex than what it appears on the surface, as the naive implementation showed. RxJS is a powerful tool to manage streams of data and can be used in this instance to create a reliable real time search.
