Chapter 12. Embedded data and associations

This chapter covers

  • Using embedded records and associations to create relationships between models
  • Reducing duplicative code by creating a service
  • Understanding the differences between one-way and two-way associations

In chapter 11, we examined the new requirements of a market-based pivot that significantly enhanced and extended the features of Brushfire. We applied our frontend-first approach to the requirements. Using this approach compelled us to make design decisions in advance of implementing the backend, thereby reducing the likelihood of second-guessing based on inadequate information. Essentially, we wanted to eliminate questions that begin with “I didn’t know that....” Our work in chapter 11 produced a set of documented decision points in the API Reference, wireframes, and a functioning frontend with simulated backend responses.

Up to this point in our application, we haven’t had to concentrate on the organization of models and the underlying database of Brushfire. We began the book by creating and storing URLs of YouTube videos in a video model, first using an array on the frontend to simulate a database and then storing the array using Sails’ NoSQL in-memory database. We expanded the initial model/database implementation to include storing information about users in a user model, and then stored user records in a PostgreSQL database. With our new feature requirements, we now need to take the added models to support those features and relate them to each other. For example, a user record will be associated with the tutorial records it creates. This chapter will examine our new models, their relationships, and how to fulfill the requirements of requests using Sails associations.

12.1. Obtaining the example materials for this chapter

If you’ve been following along in chapter 11 with an existing project, you can continue to use that project in this chapter. If you want to start from this chapter and move forward, clone the following repo: https://github.com/sailsinaction/brushfire-ch11-end. After cloning the repo, install the Node module dependencies via npm install. You’ll also want to add the local.js file you created in chapter 11. In Sublime, create a new file in brushfire/config/local.js, and add the following code to it.

Example 12.1. Adding to the local.js file 
module.exports.blueprints = {
  shortcuts: true,
  prefix: '/bp',
};

module.exports.connections = {
  myPostgresqlServer: {
    adapter: 'sails-postgresql',
    host: 'localhost',
    database: 'brushfire'
  }
};

module.exports.mailgun =  {
  apiKey: 'ADD YOUR MAILGUN API KEY HERE',
  domain: 'ADD YOUR MAILGUN DOMAIN HERE',
  baseUrl: 'http://localhost:1337'
};

12.2. Understanding relationships between data

We’ve established that a model is an abstract representation of a noun, like a User. You probably recognize figure 12.1 by now.

Figure 12.1. A model consists of attributes , methods , settings , and an adapter .

Each model has details about the types of attributes it contains; the methods used to find, create, update and destroy records; and information related to where and how the records are stored. You use models to organize, access, and modify records whose values are part of the requirements of requests generated from the frontend.

12.2.1. Brushfire models after the pivot

After the pivot, the Brushfire repo now contains the following models:

  • user—Holds values about a user
  • tutorial—Holds values about a tutorial
  • video—Holds values about a video
  • rating—Holds values about a rating
  • chat—Holds values about a chat message (which you’ll implement in chapter 14)

You’ll explore these models when you use them to implement the remaining Brushfire features. Post-pivot, you also have requests that require records from multiple models. For example, the tutorials-detail page contains attributes from four different models, as shown in figure 12.2.

Figure 12.2. The tutorials-detail page contains information from the user, tutorial, video, and rating models.

Let’s look at how you can set up relationships to more effectively access and modify the attributes of records across multiple models.

12.2.2. Relationships between models

Figure 12.3 illustrates some of the inherent relationships between Brushfire models.

Figure 12.3. Models have relationships that go in different directions, as depicted by the arrows in the figure.

Each of the relationships in figure 12.3 shares two important characteristics—the direction of the reference between models and the number of records referenced:

  • Direction (one-way or two-way reference)— A relationship has a two-way reference if both models require the ability to find, create, update, or destroy records in each model. For example, between the user and tutorial models, there’s a need for a user to find all tutorials created by that user and for the tutorial to find the user who created it. This relationship requires a two-way reference.
  • Quantity (one or many)— Does the related model have a relationship with one record or multiple records? Between the user and tutorial models, the user can own multiple tutorials, but a tutorial can be owned by only one user.

You know that relationships exist, but how are they implemented between models? You have three ways to create relationships in models and their records in Sails:

  • Embedding values— Embed values from one model’s record into another model’s record and then maintain each embedded value for each record. For example, the tutorial model could embed the sort order of the video records for a tutorial as an array of dictionaries. This works as long as you don’t need to query on the embedded value of the record. We’ll expand on this in the next section.
  • One-way association reference— Use Waterline associations to create a one-way reference association between two models and then maintain each reference.
  • Two-way association reference— Use Waterline associations to create a two-way referenced association between two models and maintain a single reference for both associations.

You’ll implement all three types of relationships in order to fully understand how they work and when to use them.

12.3. Associating data using embedded JSON

Remember, embedding means you’re adding the actual values of one record to another record and then manually syncing those values on both records when they change. You’re going to implement the backend requirements for the create-tutorial page to create all three types of relationships. First, look at the wireframe for the view in figure 12.4.

Figure 12.4. The create-tutorial page contains two parameters and two requests.

The create-tutorial request requirements from the API reference are shown in figure 12.5.

Figure 12.5. The create-tutorial request reference contains a description , the method and path of the request , backend notes , incoming parameters , the target action , the response type , and the response .

The request expects you to take the incoming title and description parameters and create a new tutorial. You need to associate the new tutorial record with the currently authenticated user and then respond with the new tutorial id as JSON. So your task on the backend is to create a record in the tutorial model that embeds the user who created it, as well as to update a record in the user model that embeds the new tutorial.

12.3.1. Setting up an embedded relationship

First, you’ll set up the models with attributes that will hold your embedded values. In Sublime, open brushfire/api/models/Tutorial.js and brushfire/api/models/User.js. The models should look similar to figure 12.6.

Figure 12.6. The user model contains a tutorials attribute that will hold the embedded array of tutorial dictionaries as JSON and an owner attribute that will hold the embedded username dictionary as JSON.

The tutorials attribute in the user model will keep track of the tutorials created by the currently authenticated user-agent. This will enable you to query on the user model to obtain a list of the tutorials created by that user. The user model can contain one or more tutorials to be stored as an array of dictionaries in JSON. The owner attribute in the tutorial model will keep track of which user created the tutorial.

12.3.2. Creating a record with embedded JSON

You now must implement the backend createTutorial action to fulfill the requirements of the create-tutorial request. When a user-agent fills out the form on the create-tutorial page and clicks the Save button, the frontend makes an Angular AJAX POST request to /tutorials, triggering the createTutorial action of the tutorial controller. Let’s start adding code to the action. In Sublime, open brushfire/api/controllers/ TutorialController.js, and add the code to validate the incoming parameters in the existing createTutorial action, as shown in the following listing.

Example 12.2. The createTutorial action—validating incoming parameters

You use Lodash’s _.isString() method to ensure that the incoming parameters are both defined and of the proper type. Next, you must query for the user using the currently authenticated user-agent. In Sublime, add the following to the create-Tutorial action.

Example 12.3. The createTutorial action—find the currently authenticated user

Now you can create the tutorial using the incoming parameters and assign the embedded username attribute with the username found in the user record. In Sublime, add the following code to the createTutorialaction action.

Example 12.4. The createTutorial action—create the tutorial and update owner

You’ll complete the action by updating the user record with the embedded tutorial array of dictionaries, and then you’ll return the id of the new tutorial as JSON. In Sublime, add the following code.

Example 12.5. The createTutorial action—update the tutorials attribute

Notice that you copy the unique record id, createdAt date, and updatedAt date from the tutorial record as elements of your dictionary and embedded array. This combination of a separate tutorial model with an embedded JSON array in the user model is working, but there are pitfalls to this approach.

12.3.3. Populating embedded data

Turn your attention to the edit-tutorial page shown in figure 12.7.

Figure 12.7. The edit-tutorial page contains two parameters and two requests.

Unlike the create-tutorial page, the edit-tutorial page requires that you gather some tutorial attributes for the view. You’ll first alter the existing editTutorial action of the PageController to find the tutorial by using the incoming tutorial id parameter, and you’ll find the user of the currently authenticated user-agent via the session userId.

Tip

You use the unary operator (+) to coerce the tutorial id into a number. You do this to ensure that the id you provide is a number and not a string.

In Sublime, open brushfire/api/controllers/PageController.js, and replace the fake tutorials dictionary with the Tutorial.findOne() method, similar to the next listing.

Example 12.6. The editTutorial action—find the tutorial and authenticated user

After finding the tutorial and the currently authenticated user-agent, you’ll check whether that user is the owner of the tutorial. In Sublime, add the following code.

Example 12.7. The editTutorial action—check for the owner and display or redirect

If the user is the owner, you render the edit-tutorial page with the appropriate locals. If not, the browser is redirected back to the tutorial. Next, you implement the edit-tutorial request that actually updates the tutorial.

12.3.4. Updating a record with embedded data

Now that you have the edit-tutorial page displayed with the appropriate tutorial title and description, let’s look at the requirements for the edit-tutorial request from the API Reference, as shown in figure 12.8.

Figure 12.8. The edit-tutorial request reference contains a description , the method and path of the request , the backend notes , incoming parameters , the target action , the response type , and the response .

The request requires you to update the tutorial with the incoming title and description parameters and then to respond with a 200 status. In Sublime, open brushfire/api/controllers/TutorialController.js, and begin implementing the action by adding the following code to validate the incoming parameters in the existing create-Tutorial action.

Example 12.8. The updateTutorial action—validating incoming parameters

Next, you’ll find and update the title and description of the tutorial.

Example 12.9. The updateTutorial action—update the tutorial

Because the owner of the tutorial doesn’t change, you don’t need to update the tutorial.owner embedded attribute of the tutorial model. But because the tutorial has been modified, you need to update the user.tutorial attribute on the user record. You’ll start the process of updating this embedded tutorials array in the following listing.

Example 12.10. The updateTutorial action—update the user’s tutorials attribute

This is where things get a bit more verbose. In order to update the embedded user.tutorials array with the updated title and description parameters, you first need to gather all users in memory and search through each user’s tutorials attribute for a matching tutorial id. The function that loops through the array of user dictionaries needs to be asynchronous because it also contains User.update(),which is also asynchronous. This prevents User.update() from executing before the loop has a chance to iterate through all the users, as illustrated in figure 12.9.

Figure 12.9. Using async.each() allows you to iterate over the users to find the tutorial before executing User.update().

Once you determine the matching tutorial, you can update user.tutorials with it. Although this works, it seems like a lot of extra work to maintain both sides (tutorial and user models) of the embedded attributes. There’s also another issue related to server memory. As Brushfire grows more popular, there could be thousands—even millions—of users. Given enough users, server memory could be exhausted by trying to load each user’s tutorials array. You need another way of handling relationships of large datasets between models. Fortunately, Sails provides another approach through Waterline associations.

Note

Situations will arise when an embedded attribute is superior to an association. Later in this chapter, you’ll use an embedded array to track the sort order of videos. A good candidate for an embedded value is one in which the information you’re embedding won’t be queried outside the context of the model where it’s being embedded.

12.4. Understanding Sails associations

Figure 12.10 illustrates the embedding approach used in the last section by copying a record from a model and embedding it into a record from a different model.

Figure 12.10. An illustration of embedding a record from one model into the record of another model. The user record’s username attribute is embedded in the tutorial record’s owner attribute . The entire tutorial record is embedded into the user record’s tutorials attribute .

This requires you to maintain information that might change in two different places: in the model and in the embedded location. Associations set up a reference between models instead of copying and embedding parts of a record from another model, as illustrated in figure 12.11.

Figure 12.11. An illustration of associating a record from one model into the record of another model. The user record’s id value is now referenced in the tutorial record’s owner attribute . The tutorial record’s id is now referenced in the user record’s tutorials attribute . Note that the user record’s tutorials attribute won’t be displayed as an array. To obtain what’s in the tutorials attribute, you’ll use the populate method later in this chapter.

Tip

An association can be thought of as similar to a shortcut or alias to a file or folder in OS X’s Finder or Windows’ File Explorer. The shortcut, when clicked, opens a file or folder without having to have a copy of the contents of the file or folder.

But how are the references turned into what is referenced? How do you exchange a reference for the actual values of the record being referenced? We’re getting a little ahead of ourselves, but later in this chapter we’ll show you how to use the populate method to populate a reference with the actual record values the reference is pointing to. What’s important now is to understand that with associations any update to a referenced record’s attributes (other than the id that references them) is automatically in sync with the record that’s referencing it. If there was ever a need for an illustration for a tongue-twisting concept, this is it! As shown in figure 12.12, if you update the title of the tutorial record, the tutorials reference in the user model remains the same. When you exchange the tutorials attribute for tutorial, the updated title is displayed.

Figure 12.12. Before the title of the tutorial record is changed, if you exchange the tutorials attribute reference for the actual tutorial record values, you get a title of “The best of Douglas Crockford on JavaScript.” After the title of the tutorial record is changed , exchanging the tutorials attribute reference for the actual tutorial record values will produce a title of “The best of Nikola Tesla on Electricity.” Thus, the reference remains in sync after changes.

Referencing another record through an association eliminates the requirement to update both records in the relationship. In order to use the association, you need to configure your models to use them. You’ll do that in the next section.

12.4.1. Configuring an association between two models

Configuring an association is a simple process of adding an attribute to a model that tells Sails the following:

  • The model to associate to
  • Whether to associate one record, also called a model association, or many records, also called a collection association
  • In the case of a collection, whether it’s a one-way or a two-way association

For example, in figure 12.13, let’s take a look at the user and tutorial models, specifically the attributes related to the relationship that’s configured.

Figure 12.13. The user model contains a tutorials attribute that configures a collection association between the user and tutorial models. The tutorial model contains an owner attribute that configures a model association between the tutorial and user models.

The user model employs the same attribute name you used with the embedded technique: tutorials. You want to set up an associated reference to all tutorials created by this user. Because this references multiple records, a collection association will be used. Because this is a collection association, you can configure it to be a one-way or two-way reference. We’ll explore two-way references later in this section. For now, you’ll use a one-way reference that doesn’t require any further configuration. In Sublime, open brushfire/api/models/User.js, and add the tutorials association attribute shown in the next listing.

Example 12.11. The user model’s tutorials collection association attribute

Now let’s create the association in the tutorial model. In Sublime, open brushfire/ api/models/Tutorial.js, and add the owner association attribute shown here.

Example 12.12. The tutorial model’s owner association attribute

The tutorial model employs the same attribute name you used with the embedded technique: owner. You want to set up an associated reference to a single user, so you’ll use a model association. At first, this can be confusing. But all this means is that the owner attribute, when populated, will produce a single record: a user record. Let’s head back to the createTutorial action in the tutorial controller and start using these associations.

12.4.2. Using .add(), .remove(), and .save()

When a user-agent creates a tutorial, you have two references to keep in sync: one for the user model and one for the tutorial model. With a collection association, you use .add() to add a reference and .remove() to remove a reference. With a model association, you add the id of the record you want to associate. In Sublime, open brushfire/api/controllers/TutorialController.js, and add the following code to the createTutorial action.

Example 12.13. Refactored createTutorial action using associations

When a user creates a tutorial, you first add the id of that user to the owner attribute of the currently authenticated user-agent. This provides a reference to the tutorial record from the user record. You then use .add() to add the id of the newly created tutorial to the user’s tutorials attribute. This provides a reference to the user record from the tutorial record. Now that you’ve set up the references, you can use the .populate() method to add a snapshot of the record or records referenced by a particular association at the time of the query. Head back to the createTutorial action in Sublime at brushfire/api/controllers/TutorialController.js, and add the following code.

Example 12.14. Using .populate() to transform a referenced link

Now, when a user creates a tutorial, the terminal window will contain a log similar to figure 12.14.

Figure 12.14. Before using .populate(), the user and tutorial records contain references whose value is the id of the associated record. After using .populate(), those references are populated with the tutorial record and the username attribute of the user record.

With associations, you have the benefits of the embedded technique without the inherent syncing requirements and memory challenges.

12.4.3. Using via to create a two-way association

Earlier, we discussed the possibility of using a two-way reference with collection associations. When a collection association has a second model that points back to it, you have the option of using the via parameter. Using via reduces the number of references you need to manage in the association. In our current example, the user model has a collection referencing the tutorial model, and the tutorial model has a reference to the user model. If you add the via parameter to the user’s collection model when you update one reference, then the other reference is automatically in sync. In Sublime, open brushfire/api/models/User.js, and add the following parameter to the user model.

Example 12.15. Using the via parameter in the user’s tutorials collection attribute

12.4.4. Refactoring an action to use associations

To understand the advantages of via, let’s refactor the createTutorial action to utilize via by opening brushfire/api/controllers/TutorialController.js in Sublime, and adding the code in the following listing.

Example 12.16. Refactoring the createTutorial action to take advantage of via

Previously, when a user created a tutorial, you updated the owner attribute in the tutorial model and the tutorials attribute in the user model. By adding the via parameter, you can update either reference and the other reference will be kept in sync automatically. So, in listing 12.16, when you updated the owner parameter reference of the tutorial model, there was no need to update the tutorials parameter reference in the user model.

12.4.5. Using .populate()

The editTutorial action of the page controller determines whether the currently authenticated user-agent is the creator of the tutorial. Within the action, you compare the results of a query from the user model to a query from the tutorial model. Because you’re using associations, the value of tutorial.owner.username will be null. This is because the owner property is now an association model reference and returns the id of a user record and not the entire record. You need to replace the value of tutorial.owner from a user record id to a snapshot of the record itself. In Sublime, open brushfire/api/controllers/PageController.js, and add the following code to the editTutorial action.

Example 12.17. Refactoring the editTutorial action to take advantage of associations

When you populate the owner association reference, the value for foundTutorial.owner will be replaced with a snapshot of the user record. Your comparison of the currently authenticated user-agent username with the username of the tutorial.owner.username will now work as required.

Now you can remove all the code for the updateTutorial that involved propagating the changes to the embedded tutorials array attribute in the user model. Remember, with the association reference, you simply need to update the record that changes and all references to that record will be in sync. In Sublime, open brushfire/api/controllers/TutorialController.js, and refactor the updateTutorial action similar to the next listing.

Example 12.18. Refactoring updateTutorial action to take advantage of the association 
...
updateTutorial: function(req, res) {

  if (!_.isString(req.param('title'))) {
    return res.badRequest();
  }

  if (!_.isString(req.param('description'))) {
    return res.badRequest();
  }

  Tutorial.update({
    id: +req.param('id')
  },{
    title: req.param('title'),
    description: req.param('description')
  }).exec(function (err) {
    if (err) return res.negotiate(err);

    return res.ok();
  });
},
...

Using associations substantially eliminates a lot of the syncing overhead you’d need to do manually. You’ll use these techniques to complete more of the backend requirements of Brushfire.

12.4.6. Refactoring bootstrap.js to use associations

Post-pivot, you need to adjust the way you create test records in the bootstrap. Your needs go beyond test users and now include tutorials, videos, and ratings. The techniques used to create the test data are beyond the scope of the book. A link to a mini-tutorial for the bootstrap can be found off the main hub link at http://sailsinaction.github.io/. For now, make the following changes to brushfire/api/config/bootstrap.js in Sublime.

Example 12.19. Changing the way you create test users in bootstrap.js 
module.exports.bootstrap = function(cb) {
  var FixtureBootstrapper = require('../fixtures');
  return FixtureBootstrapper(cb);
};

Restart Sails using sails lift and navigate your browser to localhost:1337/bp/user/ find. The blueprint find shortcut is executed and three users are displayed. The user sailsinaction has tutorials, videos, and ratings. You’ll use these test records throughout the next chapters.

12.5. Using services

Much of what you do in page controller actions is transforming data into a format that the frontend expects. The tutorials-detail page is currently using a simulated tutorial dictionary that was added when you cloned the project in chapter 11. The simulated dictionary also contains the tutorial.owner attribute of the user who created the tutorial. Let’s start refactoring the tutorialDetail action of the page controller by replacing the simulated dictionary with a query to the tutorial model.

12.5.1. Example: using associations for the tutorials-detail page

You want to transition from using a simulated dictionary in the tutorialDetail action to an actual query to the tutorial model. You’ll also need to populate the owner association reference to exchange the user id with an actual user record. In Sublime, open brushfire/controllers/PageController.js, and replace the simulated dictionary in the tutorialDetail action with the following code.

Example 12.20. Replacing the simulated tutorial dictionary

Populating the owner association reference will replace the user id with a snapshot of the user record. Because the frontend requires that the owner attribute contain the username of the user who created the tutorial, you’ll transform it to contain just the username. Make sure Sails is running via sails lift, and navigate your browser to localhost:1337/tutorials/1, which should look similar to figure 12.15.

Figure 12.15. The username attribute is displayed, but the tutorials-detail page expects the createdAt and updatedAt attributes in the time-ago format .

The username is properly displayed, but the createdAt and modifiedAt attributes are not being displayed. Let’s fix that next.

12.5.2. Using a service to consolidate duplicative code

Again looking at the API Reference, you need to transform the createdAt and modified-At attributes from their current JSON timestamp format, 2015-09-27T16: 32:55.000Z, to the popular time-ago format, 2 days ago. At first, it might appear that the best thing to do is declare a function similar to the following.

Example 12.21. Formatting the createdAt date with machinepack-datetime

This works, but it violates one of our internal truisms: Never declare a function inside another function unless it’s a callback or deals with recursion. Declaring a function within another function (for example, inline) in Sails can lead to issues for these reasons:

  • You don’t know when the code is going to run.
  • You don’t know if the code inside the function implementation is relying on closure scope for variables or function calls.
  • When reading the code, it completely derails the linear flow of what’s supposed to happen. It’s as if, in the book, we interrupted ourselves in the middle of a paragraph and went on a tangent for three pages about cats. Even though Chad’s mom would certainly approve, that wouldn’t make sense.

If you look through the Brushfire API Reference, you’ll find that you need to transform various dates into this format in no less than six different Brushfire actions. This violates another internal truism: If you’ve repeated code in three or more locations, it’s time to think about writing a service. But what’s a service?

A service is one or more functions declared within a dictionary that Sails globalizes, making the functions available everywhere. The project you cloned in chapter 11 already contains the new service in brushfire/api/services/DateTimeService.js, similar to the following listing.

Example 12.22. Declaring the Datetime service 
module.exports = {
  getTimeAgo: function (options) {
    var Datetime = require('machinepack-datetime');

    var niceTimeAgoString = Datetime.timeFrom({
      toWhen: Datetime.parse({
        datetime: options.date
      }).execSync(),
      fromWhen: new Date().getTime()
    }).execSync();

    return niceTimeAgoString;
  }
};

The name of the file will become the globalized name of the service. You also add the suffix Service to the name to differentiate it from other globalized dictionaries such as models like User.js. Our preferred method signature for services that contain synchronous functions is function(options) {}. Our preferred method signature for asynchronous functions is function (options, cb){}. In Sublime, open brushfire/controllers/PageController.js, and add the code to transform the createdAt and updatedAt attributes using the Datetime service, as shown here.

Example 12.23. Using the Datetime service

Restart Sails using sails lift and navigate your browser to localhost:1337/tutorials/1, and you should see something similar to figure 12.16.

Figure 12.16. The tutorials-detail page with the username , createdAt , and updatedAt properties formatted properly

The username, createdAt, and updatedAt properties are now in the format the frontend expects.

12.6. Summary

  • You can create relationships between models using embedded attributes or association attributes.
  • Model relationships have two important characteristics: the direction of the relationship and the quantity of the relationship.
  • The Sails ORM uses embedded attributes, models, and collections via associations.
  • You should use an embedded relationship only if the attribute embedded won’t need to be queried outside the model.
..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset