Reducing the size of data as much as possible

We all know that Qlik Sense loads all the data loaded using the script in RAM. The more the data you have, the more RAM is required, along with more time to transfer the data to the RAM. So, as a script optimization process, you should look at possible ways to reduce the size of the data.

Some ways to reduce the size of the data are explained in the following list:

  • Remove unwanted fields: Often, it is seen that the developer loads all the fields available in the table in the Qlik Sense data model. This is fine when the data is small, and the numbers of columns are few, but when the data is huge, and the columns are many, the developer should remove or comment the unwanted fields from the data model. This helps to quicken the data loading process.
  • Reduce distinct values: Qlik Sense stores only the distinct values in the RAM, so when you have more distinct values in data, your data size is more. Thus, you should try to decrease distinct data. For example, your source system may have a timestamp field, but for your analysis, you do not require the time. In such cases, you can remove the time from the timestamp field, which will reduce the size of the data.
  • Use the AutoNumber function: While implementing the link table in the data model, we saw that we need to create the composite key, which is the concatenation of multiple fields. These composite keys are distinct in nature and contain text values, but they are needed only to link two or more tables with each other. In such cases, we can convert the text value of the composite key with the number using the AutoNumber() function.

The AutoNumber() function allocates each unique distinct value to the unique value of the text field. This way, you can reduce the data size, because the text value requires more space then numeric values.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset