<a href="https://docs.bullmq.io/" target="_blank">BullMQ</a>, A fast and efficient Job queuing solution for scaling your microservices. It is a <a href="https://nodejs.org" target="_blank">Node.js</a> library that is built on top of <a href="https://redis.io/" target="_blank">Redis</a> that helps to solve many problems related to microservices architecture.

As we already discussed, it's a job queuing solution for scaling your microservices. Bull MQ is used by many organizations, whether they are small or big. Here are a few of them, Microsoft, Vendure, Data Wrapper and CURRI.

It is used in microservices for communication purposes. So by using this technology in your microservice architecture, you can build a communication in between them to transfer the data, also known as messages.

So now I assume that you have understood what exactly is BullMQ, Now, before moving to the implementation, we should first understand what exactly is Microservice Architecture. And why do we need to build the communication in between them?

<a href="https://www.youtube.com/@CodeArchitectureWithRahul" target="_blank">Microservice architecture</a> is architectural pattern in which we have multiple loosely coupled services. In very simple terms. If you have created any server side project, You might have created it using tightly coupled services Also know that monolithic architecture.

So in monolithic architecture, what exactly we do is We build a complete project with one single code base. Which means there can have multiple services like:

Typically, these are ecommerce services and if you have created any ecommerce application, you might have written all the code for these services in one single code base, Or you can also say that you can access to all the API routes for these services from single server (http://localhost:5000) which can look something like:

Login: http://localhost:5000/v1/login

Registration: http://localhost:5000/v1/registration

Product Buying: http://localhost:5000/v1/product/purchase/123456

 This is knows as Monolithic architecture, Where you write all your code in one single folder and which runs on single server!

Whereas microservices is different, in microservice architecture, we create one server for one service, suppose we convert the about ecommerce example in Microservices then it would look like:

Product Buying: http://localhost:5002/v1/product/purchase/123456

So here you can see we have created different server for different services.

Now let's understand the 3 main components of BullMQ:

In BullMQ the concept of the queue is not different it is the same as we learnt in data structures and algorithm. In message queuing system a queue is used for storing the messages/data. It runs on the principle of first in, first out (FIFO)&nbsp;which means that the first data added to the queue will be processed first

workers in BullMQ are responsible for processing the jobs, They are capable of handling concurrency, which means they can handle more than one work at a time.

In BullMQ, the Queue events are used for managing and monitoring the state of the job. Which means they can tell you whether the job is completed or failed as well as there are many more events that you can track!

We just required 2 things for implementing BullMQ in Nodejs:

Docker: In this tutorial, we will be using Docker for containerization.

Redis Insight: A user interface that helps you visualize and optimize your data for Redis and Redis Stack.

To download Docker you can visit this link: <a href="https://www.docker.com/products/docker-desktop/" target="_blank">Download Docker Desktop</a>. Once you download docker you can run the setup and create a new account.

This is how it will look after the fresh installation! Now we are good to go to download and install out next requirement which is Redis Insight.

We need to install <a href="https://hub.docker.com/_/redis" target="_blank">Official Redis Docker Image</a>&nbsp;as BullMq internally uses Redis for communication. so just run the command and it will install redis as well as it will run redis in container!

If you don't have redis downloaded earlier then it will automatically download it. else it will directly run that in container. and if you open your docker app, you will see one container has up and running as you can see in the figure

As you can see the redis is running on port 6379. And that's if we have successfully installed and configured redis.

To download and install Redis Insight, you can visit this link: <a href="https://redis.io/insight/#insight-form" target="_blank">Download Redis Insight</a>.&nbsp;

If you see this screen then you have successfully installed Redis Insight, Now lets Add Redis Database so that we can watch all the data which will be stored inside the queue.

Click on Add Redis Database and make sure you have did the same configuration as I have did!

Why do we required Docker and Redis Insight?

Docker is the most famous platform for building Containerized applications. So basically containerized applications is something like you imagine there is a container and in that single container you can run Operating System (OS), you can create multiple containers, and you can run multiple operating systems in those containers. 

Not only operating systems, but we can just define all your required dependencies in that single container.&nbsp;If you want to share your code or project with friends or teammates, you can share the Dockerfile along with your code. They can simply run the Dockerfile, and Docker will automatically install all required dependencies, eliminating version mismatch issues.

So for example: in you project docker file, if you mentioned that this project required Nodejs v20 and you share the code base with your friend. If you friend doesn't have Nodejs v20 install but he has Nodejs v18 still he can run that project. 

So under the hood, Docker will create a container and inside of that container, it will install Nodejs V20. So now your friend has Nodejs v20! hurray

Use of Redis Insight for implementing BullMQ

Redis Insight is the user interface provided for desktop where users can&nbsp;visualize and optimize data for Redis and Redis Stack. 

It also supports Command Line (CLI) interaction, You can watch their official video on YouTube <a href="https://www.youtube.com/watch?v=3MXOvoX7Sdw" target="_blank">Bite Sized Redis: RedisInsight</a>&nbsp;also you can learn more about redis insight by visiting the <a href="https://redis.io/docs/latest/" target="_blank">official docs</a>

So now we have clearly understood what exactly is docker and redis insight. Now we can move further and set up our environment so that we can code efficiently.

What we are going to build in this tutorial?

So in this story tutorial, We have built an ecommerce microservice and we are going to set up a communication in between them using BullMQ.

User Server - Manages all users and verifications

Mail Server - Responsible for sending mails

All the users will hit request on our <code class="inline-code">Product service</code> for purchasing a single product, And the product service is isolated, So the <code class="inline-code">Product Service</code> is unaware of which user has requested to purchase a product.

Product service will try to communicate with <code class="inline-code">User service</code> to validate and verify the user, by placing the UserID, JWT token or any other Unique identifier in the <code class="inline-code">user-verification-queue</code> and the User Service will listen for the event happening in that particular queue. Once the token in placed in the queue, <code class="inline-code">User service</code> will extract the token and verify the user.

Once verification is completed the message will be sent back to Product Service with the appropriate data. Now Product service has all the required information about the user.

Product Service will save the order to the database. Once order is saved in database, the system or product service wants to send Mail Notification on user's Mail Id.&nbsp;&nbsp;

Now the Product service will Communicate with mail service for sending mails. Similarly we will create a new Queue named <code class="inline-code">mail-queue</code> and&nbsp;&nbsp;will place data inside the queue, This data might contain user information like Mail Id, as well as the product information which the user has purchased.&nbsp;

The Mail Service will keep listening for event happening in that&nbsp;<code class="inline-code">mail-queue</code>&nbsp;and once the data is placed inside the queue, the mail service will take the data&nbsp;from that queue and the service will send mail to that particular user, Informing that the order was placed successfully.

Implementing 3 Microservices with help of BullMQ in NodeJs&nbsp;&nbsp;

Project structure before installing all dependencies

Before installing any dependencies, your project structure must look something like this.

You must install <code class="inline-code">Express.js</code> and <code class="inline-code">BullMQ</code>&nbsp;in each folder (Product, User and Mail service), You can install those dependencies by using these commands:

Once you installed all required dependencies, your folder structure, or your project structure, must look something like this:

Project structure after installing all dependencies

We will first create a basic server which will run on Port <code class="inline-code">5001</code>

As in this story tutorial, we are not going to set up any database. So for database operation, we will be using our dummy database just to keep things simple.

Worker is the one who will listen to the particular queue. So we will use the Worker from BullMQ. The worker takes 3 arguments:

Name of the queue to which the worker should listen

Callback function which will run the some data gets placed inside the queue

The <code class="inline-code">job</code>&nbsp;parameter in callback will give us all the data which was in the queue! In our case we will just receive the User Id or JWT Token:

Now by using vanilla Javascript, we will simply verify the user:

I that's it!! we have successfully implemented the User Service, Let's see how to complete code looks

Explained BullMQ in the most easiest way possible using Docker & Redis Insight, create Asynchronous communication in between your microservices using BullMQ. 

BullMQ implementation in Nodejs with Redis & Docker | Efficient Job Queuing solution

In this today's&nbsp;<a href="https://ww.codearchitecture.in" target="_blank">Code Architecture</a> story we will cover all NodeJS interview questions that were asked for junior and senior developers in their interviews in year 2024. We have collected these question from students and senior developers working as MERN developer or Backend <a href="https://www.naukri.com/node-js-developer-jobs" target="_blank">NodeJS Developer</a> at various startups and MNCs. These questions target packages ranging from 3 LPA to 20 LPA

We have divided the questions in categories so that it will be easy for you to identify which question belongs to which topics!

<a href="https://www.codearchitecture.in/stories/100-nodejs-interview-questions-most-interviewers-ask-in-2024#NodeJS-Interview-Basics-and-Core-Concepts" target="_self">Basics and Core Concepts</a>

<a href="https://www.codearchitecture.in/stories/100-nodejs-interview-questions-most-interviewers-ask-in-2024#NodeJS-Interview-Mid-level-advance-Concepts" target="_self">Mid level advance Concepts</a>

<a href="https://www.codearchitecture.in/stories/100-nodejs-interview-questions-most-interviewers-ask-in-2024#Event-Driven-Architecture" target="_self">Event-Driven Architecture</a>

<a href="https://www.codearchitecture.in/stories/100-nodejs-interview-questions-most-interviewers-ask-in-2024#Streams-and-Buffers" target="_self">Streams and Buffers</a>

<a href="https://www.codearchitecture.in/stories/100-nodejs-interview-questions-most-interviewers-ask-in-2024#File-System-operation" target="_self">File System operation</a>

<a href="https://www.codearchitecture.in/stories/100-nodejs-interview-questions-most-interviewers-ask-in-2024#Process-Management" target="_self">Process Management</a>

<a href="https://www.codearchitecture.in/stories/100-nodejs-interview-questions-most-interviewers-ask-in-2024#Network-and-HTTP" target="_self">Network and HTTP</a>

<a href="https://www.codearchitecture.in/stories/100-nodejs-interview-questions-most-interviewers-ask-in-2024#Timers" target="_self">Timers</a>

<a href="https://www.codearchitecture.in/stories/100-nodejs-interview-questions-most-interviewers-ask-in-2024#Error-Handling" target="_self">Error Handling</a>

<a href="https://www.codearchitecture.in/stories/100-nodejs-interview-questions-most-interviewers-ask-in-2024#Performance-Optimization" target="_self">Performance Optimization</a>

<a href="https://www.codearchitecture.in/stories/100-nodejs-interview-questions-most-interviewers-ask-in-2024#Security" target="_self">Security</a>

<editorjs-style id="NodeJS-Interview-Basics-and-Core-Concepts">NodeJS Interview Basics and Core Concepts</editorjs-style>

<editorjs-style id="NodeJS-Interview-Mid-level-advance-Concepts">NodeJS Interview&nbsp;Mid level advance Concepts</editorjs-style>

<editorjs-style id="Event-Driven-Architecture">NodeJS interview question on&nbsp;Event-Driven Architecture</editorjs-style>

<editorjs-style id="Streams-and-Buffers">NodeJS interview question on Streams and Buffers</editorjs-style>

<editorjs-style id="File-System-operation">NodeJs interview questions on File System operation</editorjs-style>

<editorjs-style id="Process-Management">NodeJS Interview questions on Process Management</editorjs-style>

<editorjs-style id="Network-and-HTTP">NodeJS interview question on Network and HTTP</editorjs-style>

<editorjs-style id="Timers">NodeJS interview questions on&nbsp;Timers</editorjs-style>

<editorjs-style id="Error-Handling">NodeJS interview ques</editorjs-style>tions on&nbsp;Error Handling

<editorjs-style id="Performance-Optimization">NodeJS interview questions on&nbsp;Performance Optimization</editorjs-style>

<editorjs-style id="Security">NodeJS interview question on&nbsp;Security</editorjs-style>

100 Node.js Interview Questions Most Interviewers Asked in 2024, Covering All Concepts, Handpicked from Students & Experienced Senior backend Developers

Latest Stories

Spotlight

Top Picks

Exclusive

Explore by Categories

Learn Coding

Projects

System Design

Computer Science

Jobs

Technology

University News

Market Trends

Trusted by over satisfied Students and Developers

500+

500+

500+

500+

500+