Database Adapters

Moleculer framework has an official set of DB adapters. Use them to persist your data in a database.

Database per service

Moleculer follows the one database per service pattern. To learn more about this design pattern and its implications check this article. For multiple entities/tables per service approach check FAQ.

Возможности

• default CRUD actions
• cached actions
• pagination support
• pluggable adapter (NeDB is the default memory adapter for testing & prototyping)
• official adapters for MongoDB, PostgreSQL, SQLite, MySQL, MSSQL.
• fields filtering
• populating
• encode/decode IDs
• entity lifecycle events for notifications

Try it in your browser!

Base Adapter

Moleculer’s default adapter is based on NeDB. Use it to quickly set up and test you prototype.

Only use this adapter for prototyping and testing. When you are ready to go into production simply swap to Mongo, Mongoose or Sequelize adapters as they all implement common Settings, Actions and Methods.

Установка

$ npm install moleculer-db --save

Первые шаги

"use strict";

const { ServiceBroker } = require("moleculer");
const DbService = require("moleculer-db");

const broker = new ServiceBroker();

// Create a DB service for `user` entities
broker.createService({
    name: "users",

    // Mixin DB service into (current) 'users' service
    mixins: [DbService],

    settings: {
        fields: ["_id", "username", "name"],
        entityValidator: {
            username: "string"
        }
    },

    afterConnected() {
        // Seed the DB with ˙this.create`
    }
});

broker.start()

// Create a new user
.then(() => broker.call("users.create", {
    username: "john",
    name: "John Doe",
    status: 1
}))

// Get all users
.then(() => broker.call("users.find").then(console.log));

// List users with pagination
.then(() => broker.call("users.list", { page: 2, pageSize: 10 }).then(console.log));

// Get a user
.then(() => broker.call("users.get", { id: 2 }).then(console.log));

// Update a user
.then(() => broker.call("users.update", { id: 2, name: "Jane Doe" }).then(console.log));

// Delete a user
.then(() => broker.call("users.remove", { id: 2 }).then(console.log));

More examples can be found on GitHub

Настройки

All DB adapters share a common set of settings:

Property	Тип	Значение по умолчанию	Описание
idField	String	required	Name of ID field.
fields	Array.<String>	null	Field filtering list. It must be an Array. If the value is null or undefined doesn’t filter the fields of entities.
populates	Array	null	Schema for population. Read more.
pageSize	Number	required	Default page size in list action.
maxPageSize	Number	required	Maximum page size in list action.
maxLimit	Number	required	Maximum value of limit in find action. Default: -1 (no limit)
entityValidator	Object, function	null	Validator schema or a function to validate the incoming entity in create & ‘insert’ actions.

idField does not work with Sequelize adapter as you can freely set your own ID while creating the model.

Customization

As with all mixins, the standard merge algorithm allows you to override the defaults applied by this mixin. For example to disable an action you can set the action to false in your service.

Пример

"use strict";
const { ServiceBroker } = require("moleculer");
const DbService = require("moleculer-db");

const broker = new ServiceBroker();

broker.createService({
    name: "db-with-hooks",

    // Load DB actions
    mixins: [DbService],

    actions: {
        // Disable remove action
        remove: false,
        // Make create and insert public instead of published
        create: {
            visibility: "public",
        },
        insert: {
            visibility: "public",
        }

    }
});

Действия

DB adapters also implement CRUD operations. These actions are published methods and can be called by other services.

find

Find entities by query.

Параметры

Property	Тип	Значение по умолчанию	Описание
populate	Array.<String>	-	Populated fields.
fields	Array.<String>	-	Fields filter.
limit	Number	required	Max count of rows.
offset	Number	required	Count of skipped rows.
sort	String	required	Sorted fields.
поиск	String	required	Search text.
searchFields	String	required	Fields for searching.
query	Object	required	Query object. Passes to adapter.

Results

Type: Array.<Object> - List of found entities.

count

Get count of entities by query.

Параметры

Property	Тип	Значение по умолчанию	Описание
поиск	String	required	Search text.
searchFields	String	required	Fields list for searching.
query	Object	required	Query object. Passes to adapter.

Results

Type: Number - Count of found entities.

list

List entities by filters and pagination results.

Параметры

Property	Тип	Значение по умолчанию	Описание
populate	Array.<String>	-	Populated fields.
fields	Array.<String>	-	Fields filter.
page	Number	required	Page number.
pageSize	Number	required	Size of a page.
sort	String	required	Sorted fields.
поиск	String	required	Search text.
searchFields	String	required	Fields for searching.
query	Object	required	Query object. Passes to adapter.

Results

Type: Object - List of found entities and count.

create

Create a new entity.

Параметры

Property	Тип	Значение по умолчанию	Описание
-	-	-	-

No input parameters.

Results

Type: Object - Saved entity.

insert

Create many new entities.

Параметры

Property	Тип	Значение по умолчанию	Описание
entity	Object	-	Entity to save.
entities	Array.<Object>	-	Entities to save.

Results

Type: Object, Array.<Object> - Saved entity(ies).

get

Get entity by ID.

Параметры

Property	Тип	Значение по умолчанию	Описание
id	any, Array.<any>	required	ID(s) of entity.
populate	Array.<String>	-	Field list for populate.
fields	Array.<String>	-	Fields filter.
mapping	Boolean	-	Convert the returned Array to Object where the key is the value of id.

Results

Type: Object, Array.<Object> - Found entity(ies).

update

Update an entity by ID.

After update, clear the cache & call lifecycle events.

Параметры

Property	Тип	Значение по умолчанию	Описание
-	-	-	-

No input parameters.

Results

Type: Object - Updated entity.

remove

Remove an entity by ID.

Параметры

Property	Тип	Значение по умолчанию	Описание
id	any	required	ID of entity.

Results

Type: Number - Count of removed entities.

Methods

DB adapters also has a set of helper methods.

getById

Get entity(ies) by ID(s).

Параметры

Property	Тип	Значение по умолчанию	Описание
id	String, Number, Array	required	ID or IDs.
decoding	Boolean	required	Need to decode IDs.

Results

Type: Object, Array.<Object> - Found entity(ies).

clearCache

Clear cached entities

Параметры

Property	Тип	Значение по умолчанию	Описание
-	-	-	-

No input parameters.

Results

Type: Promise

encodeID

Encode ID of entity.

Параметры

Property	Тип	Значение по умолчанию	Описание
id	any	required	-

Results

Type: any

decodeID

Decode ID of entity.

Параметры

Property	Тип	Значение по умолчанию	Описание
id	any	required	-

Results

Type: any

_find

Find entities by query.

Параметры

Property	Тип	Значение по умолчанию	Описание
populate	Array.<String>	-	Populated fields.
fields	Array.<String>	-	Fields filter.
limit	Number	required	Max count of rows.
offset	Number	required	Count of skipped rows.
sort	String	required	Sorted fields.
поиск	String	required	Search text.
searchFields	String	required	Fields for searching.
query	Object	required	Query object. Passes to adapter.

Results

Type: Array.<Object>

List of found entities.

_count

Get count of entities by query.

Параметры

Property	Тип	Значение по умолчанию	Описание
поиск	String	required	Search text.
searchFields	String	required	Fields list for searching.
query	Object	required	Query object. Passes to adapter.

Results

Type: Number

Count of found entities.

_list

List entities by filters and pagination results.

Параметры

Property	Тип	Значение по умолчанию	Описание
populate	Array.<String>	-	Populated fields.
fields	Array.<String>	-	Fields filter.
page	Number	required	Page number.
pageSize	Number	required	Size of a page.
sort	String	required	Sorted fields.
поиск	String	required	Search text.
searchFields	String	required	Fields for searching.
query	Object	required	Query object. Passes to adapter.

Results

Type: Object

List of found entities and count.

_create

Create a new entity.

Параметры

Property	Тип	Значение по умолчанию	Описание
params	Object	-	Entity to save.

Results

Type: Object

Saved entity.

_insert

Create many new entities.

Параметры

Property	Тип	Значение по умолчанию	Описание
entity	Object	-	Entity to save.
entities	Array.<Object>	-	Entities to save.

Results

Type: Object, Array.<Object>

Saved entity(ies).

_get

Get entity by ID.

Параметры

Property	Тип	Значение по умолчанию	Описание
id	any, Array.<any>	required	ID(s) of entity.
populate	Array.<String>	-	Field list for populate.
fields	Array.<String>	-	Fields filter.
mapping	Boolean	-	Convert the returned Array to Object where the key is the value of id.

Results

Type: Object, Array.<Object>

Found entity(ies).

_update

Update an entity by ID.

After update, clear the cache & call lifecycle events.

Параметры

Property	Тип	Значение по умолчанию	Описание
params	Object	-	Entity to update.

Results

Type: Object

Updated entity.

_remove

Remove an entity by ID.

Параметры

Property	Тип	Значение по умолчанию	Описание
id	any	required	ID of entity.

Results

Type: Number

Count of removed entities.

Data Manipulation

You can easily use Action hooks to modify (e.g. add timestamps, hash user’s passwords or remove sensitive info) before or after saving the data in DB. Hooks will only run before or after actions. If you need to run your data manipulations before or after the this._create(), this._update() or this._remove() methods, you can use the Lifecycle events

Example of hooks adding a timestamp and removing sensitive data

"use strict";
const { ServiceBroker } = require("moleculer");
const DbService = require("moleculer-db");

const broker = new ServiceBroker();

broker.createService({
  name: "db-with-hooks",

  // Load DB actions
  mixins: [DbService],

  // Add Hooks to DB actions
  hooks: {
    before: {
      create: [
        function addTimestamp(ctx) {
          // Add timestamp
          ctx.params.createdAt = new Date();
          return ctx;
        }
      ]
    },
    after: {
      get: [
        // Arrow function as a Hook
        (ctx, res) => {
          // Remove sensitive data
          delete res.mail;
          delete res.phoneNumber;

          return res;
        }
      ]
    }
  }
});

const user = {
  name: "John Doe",
  mail: "[email protected]",
  phoneNumber: 123456789
};

broker.start()
  // Insert user into DB
  // Call "create" action. Before hook will be triggered
  .then(() => broker.call("db-with-hooks.create", user))
  // Get user from DB
  // Call "get" action. After hook will be triggered
  .then(entry => broker.call("db-with-hooks.get", { id: entry._id }))
  .then(res => console.log(res))
  .catch(err => console.error(err));

Populating

The service allows you to easily populate fields from other services. For example: If you have an author field in post entity, you can populate it with users service by ID of author. If the field is an Array of IDs, it will populate all entities via only one request

Example of populate schema

broker.createService({
    name: "posts",
    mixins: [DbService],
    settings: {
        populates: {
            // Shorthand populate rule. Resolve the `voters` values with `users.get` action.
            "voters": "users.get",

            // If the ID to populate is deep within the object, you can simply provide a dot-separated path to the ID to populate it.
            "liked.by": "users.get"

            // Define the params of action call. It will receive only with username & full name of author.
            "author": {
                action: "users.get",
                params: {
                    fields: "username fullName"
                }
            },
            // In case the original field shouldn't be overwritten with the populated values.  
            // The reviewer field will be added to the result containing the values 
            // resolved by the "users.get" action based on the reviewerId field.
            "reviewer": {
                field: "reviewerId",
                action: "users.get",
                params: {
                    fields: "username fullName"
                }
            },

            // Custom populator handler function
            "rate"(ids, items, rule, ctx) {
                // items argument is a mutable array containing response items
                // the resolved value from promise do not matter - items array will always be passed as populated response
                // so you should du custom populate by mutating items, if confused check implementaion:
                // https://github.com/moleculerjs/moleculer-db/blob/master/packages/moleculer-db/src/index.js#L636

                return Promise.resolve(...);
            }
        }
    }
});

// List posts with populated authors
broker.call("posts.find", { populate: ["author"]}).then(console.log);
// Deep population
broker.call("posts.find", { populate: ["liked.by"]}).then(console.log);

Recursive population is also supported. For example, if the users service populates a group field:

broker.createService({
    name: "users",
    mixins: [DbService],
    settings: {
        populates: {
            "group": "groups.get"
        }
    }
});

Then you can populate the group of a post author or liker like this:

//Recursive population
broker.call("posts.find", { populate: ["author.group"]}).then(console.log);
//Recursive deep population
broker.call("posts.find", { populate: ["liked.by.group"]}).then(console.log);

The populate parameter is available in find, list and get actions.

Lifecycle entity events

There are 6 lifecycle entity events which are called when entities are manipulated.

broker.createService({
    name: "posts",
    mixins: [DbService],
    settings: {},

    afterConnected() {
        this.logger.info("Connected successfully");
    },

    beforeEntityCreate(json, ctx) {
        this.logger.info("New entity will be created");
        json.createdAt = new Date()
        json.updatedAt = new Date()
        return json; // You must return the modified entity here
    },

    beforeEntityUpdate(json, ctx) {
        this.logger.info("Entity will be updated");
        json.updatedAt = new Date()
        return json;
    },

    beforeEntityRemove(json, ctx) {
        this.logger.info("Entity will be removed");
        return json;
    },

    entityCreated(json, ctx) {
        this.logger.info("New entity created!");
    },

    entityUpdated(json, ctx) {
        // You can also access to Context
        this.logger.info(`Entity updated by '${ctx.meta.user.name}' user!`);
    },

    entityRemoved(json, ctx) {
        this.logger.info("Entity removed", json);
    },    
});

Please note! If you manipulate multiple entities (updateMany, removeMany), the json parameter will be a Number instead of entities!

Extend with custom actions

Naturally you can extend this service with your custom actions.

const DbService = require("moleculer-db");

module.exports = {
    name: "posts",
    mixins: [DbService],

    settings: {
        fields: ["_id", "title", "content", "votes"]
    },

    actions: {
        // Increment `votes` field by post ID
        vote(ctx) {
            return this.adapter.updateById(ctx.params.id, { $inc: { votes: 1 } });
        },

        // List posts of an author
        byAuthors(ctx) {
            return this.find({
                query: {
                    author: ctx.params.authorID
                },
                limit: ctx.params.limit || 10,
                sort: "-createdAt"
            });
        }
    }
}

Mongo Adapter

This adapter is based on MongoDB.

Установка

$ npm install moleculer-db moleculer-db-adapter-mongo --save

Dependencies

To use this adapter you need to install MongoDB on you system.

Первые шаги

"use strict";

const { ServiceBroker } = require("moleculer");
const DbService = require("moleculer-db");
const MongoDBAdapter = require("moleculer-db-adapter-mongo");

const broker = new ServiceBroker();

// Create a Mongoose service for `post` entities
broker.createService({
    name: "posts",
    mixins: [DbService],
    adapter: new MongoDBAdapter("mongodb://localhost/moleculer-demo"),
    collection: "posts"
});


broker.start()
// Create a new post
.then(() => broker.call("posts.create", {
    title: "My first post",
    content: "Lorem ipsum...",
    votes: 0
}))

// Get all posts
.then(() => broker.call("posts.find").then(console.log));

Параметры

Example with connection URI

new MongoDBAdapter("mongodb://localhost/moleculer-db")

Example with connection URI & options

new MongoDBAdapter("mongodb://db-server-hostname/my-db", {
    keepAlive: 1
})

More MongoDB examples can be found on GitHub

Mongoose Adapter

This adapter is based on Mongoose.

Установка

$ npm install moleculer-db moleculer-db-adapter-mongoose mongoose --save

Dependencies

To use this adapter you need to install MongoDB on you system.

Первые шаги

"use strict";

const { ServiceBroker } = require("moleculer");
const DbService = require("moleculer-db");
const MongooseAdapter = require("moleculer-db-adapter-mongoose");
const mongoose = require("mongoose");

const broker = new ServiceBroker();

// Create a Mongoose service for `post` entities
broker.createService({
    name: "posts",
    mixins: [DbService],
    adapter: new MongooseAdapter("mongodb://localhost/moleculer-demo"),
    model: mongoose.model("Post", mongoose.Schema({
        title: { type: String },
        content: { type: String },
        votes: { type: Number, default: 0}
    }))
});


broker.start()
// Create a new post
.then(() => broker.call("posts.create", {
    title: "My first post",
    content: "Lorem ipsum...",
    votes: 0
}))

// Get all posts
.then(() => broker.call("posts.find").then(console.log));

Параметры

Example with connection URI

new MongooseAdapter("mongodb://localhost/moleculer-db")

Example with URI and options

new MongooseAdapter("mongodb://db-server-hostname/my-db", {
    user: process.env.MONGO_USERNAME,
    pass: process.env.MONGO_PASSWORD
    keepAlive: true
})

Connect to multiple DBs

If your services are running on separate nodes and you wish to connect to multiple databases then you can use model in your service definition. On the other hand, if your services are running on a single node and you wish to connect to multiple databases, you should define the schema that will make multiple connections for you.

More Mongoose examples can be found on GitHub

Sequelize Adapter

SQL adapter (Postgres, MySQL, SQLite & MSSQL) for Moleculer DB service with Sequelize.

Установка

$ npm install moleculer-db-adapter-sequelize --save

You have to install additional packages for your database server:

# For SQLite
$ npm install sqlite3 --save

# For MySQL
$ npm install mysql2 --save

# For PostgreSQL
$ npm install pg pg-hstore --save

# For MSSQL
$ npm install tedious --save

Первые шаги

"use strict";

const { ServiceBroker } = require("moleculer");
const DbService = require("moleculer-db");
const SqlAdapter = require("moleculer-db-adapter-sequelize");
const Sequelize = require("sequelize");

const broker = new ServiceBroker();

// Create a Sequelize service for `post` entities
broker.createService({
    name: "posts",
    mixins: [DbService],
    adapter: new SqlAdapter("sqlite://:memory:"),
    model: {
        name: "post",
        define: {
            title: Sequelize.STRING,
            content: Sequelize.TEXT,
            votes: Sequelize.INTEGER,
            author: Sequelize.INTEGER,
            status: Sequelize.BOOLEAN
        },
        options: {
            // Options from https://sequelize.org/docs/v6/moved/models-definition/
        }
    },
});


broker.start()
// Create a new post
.then(() => broker.call("posts.create", {
    title: "My first post",
    content: "Lorem ipsum...",
    votes: 0
}))

// Get all posts
.then(() => broker.call("posts.find").then(console.log));

Параметры

Every constructor arguments are passed to the Sequelize constructor. Read more about Sequelize connection.

Example with connection URI

new SqlAdapter("postgres://user:[email protected]:5432/dbname");

Example with connection options

new SqlAdapter('database', 'username', 'password', {
    host: 'localhost',
    dialect: 'mysql'|'sqlite'|'postgres'|'mssql',

    pool: {
        max: 5,
        min: 0,
        idle: 10000
    },

    noSync: true // If true, the model will not be synced by Sequelize

    // SQLite only
    storage: 'path/to/database.sqlite'
});

More Sequelize examples can be found on GitHub