• If you are citizen of an European Union member nation, you may not use this service unless you are at least 16 years old.

  • You already know Dokkio is an AI-powered assistant to organize & manage your digital files & messages. Very soon, Dokkio will support Outlook as well as One Drive. Check it out today!

View
 

DataStore-Finding With Queries

Page history last edited by Maurits Lamers 13 years, 6 months ago

 

Finding all records of a specific type is useful, but usually you need to be more selective.  When you want to search for records in your store with various conditions attached, you must make use of queries instead.

 

A query is simply an object that describes the conditions of a search.  You construct a query object using one of the helper methods defined on SC.Query.  You can perform the actual search by passing the query to SC.Store.find(), just like before.  For example, this is how you could find all records loaded into memory with a firstName of “John”:

 

var query = SC.Query.local(MyApp.Contact, ‘firstName = {name}’,{ name: ‘John’ });

var contacts = MyApp.store.find(query);

 

Other than the fact that you are passing a query object instead of a record type, using find() in this way works exactly as we described it in the previous section.  Just like before, find() returns a record array instance that will eventually populate with the results of your search as they come back from the server.  Also just like before, the returned record array will usually be “live”; updating automatically as in-memory data changes.

 

The real magic in using queries is in building queries themselves.  Queries can be used express a wide array of conditions to support all the different ways they will be used.  Learning how to find with queries, therefore, is really all about learning about queries themselves.

 

Anatomy of a Query

 

All queries have at least two properties: a recordType (or recordTypes) and a location.  Local queries may also specify additional conditions that should be used to filter and order the results.  Beyond these predefined properties, you can also add any extra properties you want to your query for use by your data source.

 

 

 

Location 

 

Each query object includes a location property which tells the DataStore which services should be used to perform the search.  Presently, the framework only supports two values for this property: SC.Query.LOCAL and SC.Query.REMOTE.

 

Local queries search data hashes that have been loaded into memory only.  While your data source may still load data into memory related to this query, the results will only reflect client-side data—not data that only resides on the server.  Querying against known, in-memory data allows these queries to be quick, responsive and amenable to conditions and ordering (see below).  Most queries you create will be local.  Local queries are built with the SC.Query.local builder method: 

 

// find all local records loaded into memory

var localRecords = SC.Query.local(...); 

 

Remote queries will expect the search to be performed on the server, and their sole purpose is to load the results into client-side memory.  If the ability is present in your DataStore, these queries can be used to load data incrementally from your server in cases where loading the whole of an extremely large data set would cause application performance to degrade.  Because these queries are performed before data is loaded into memory, you cannot apply conditions and sorting to them.  Remote queries will be comparably rare in your applications.  Remote queries are built with the SC.Query.remote builder method: 

 

// find all records remotely on the server. 

var remoteRecords = SC.Query.remote(...);

 

If you ever want to set the location of a query manually, you can define a query using the broader create() method, or even subclass SC.Query using extend() if you like.  Just make sure you define the “location” and “recordType” (or “recordTypes” if you are using an array) properties.

 

 

Record Types

 

The recordType (or recordTypes) tells the DataStore framework which records may fall into the result set for the query.  Any record type you name on a query will match records of that type and any subclassed records.  For example, if you create a query scoped to a recordType of MyApp.Contact, and you have two subclasses of MyApp.Contact - MyApp.Person and MyApp.Company - then the query will match Contact, Person, and Company records.  By extension, if you want to match all records in the store, just set recordType to the parent record object - SC.Record - from which all of your records are created.

 

// return all records.

var query = SC.Query.local(SC.Record);

 

// return all Contact-type records.

var query = SC.Query.local(MyApp.Contact);

 

If you want to match records of several different types, you can submit an array of record-types instead of just one. (TODO: Add an example of this.)  In order to avoid returning a large number of unneeded records, and paying the performance overhead of doing so, it is best to match against the narrowest record type or types possible.

 

Both local and remote queries require a record type be passed to them.

 

Fleshing Out Local Queries

 

Once a set of records has made it into local memory, the SC.Query class allows you to bring a simplified SQL-like language to bear on them.  This query language allows you to apply conditions, to separately define parameters, and to order the results by one or more record attributes - fulfilling the same functions as SQL's "WHERE" and "ORDER BY" statements.

 

Conditions

 

Conditions are passed into your local query as a string.  Simple conditions can be passed into simple queries as the query's second property, like so: 

 

// find all Contacts who are male. 

var localMenQuery = SC.Query.local(MyApp.Contact, 'gender = "male"');

 

Just like in SQL, you can get quite fancy with these conditions.  A full primer on the SproutCore Query Language can be found in the DataStore section.

 

Parameters 

 

The SC.Query class allows you to define conditions and a hash of their parameters separately.  Do so as follows:

 

// find all Contacts who are male. 

var localMenQuery = SC.Query.local(MyApp.Contact, 'gender = {gender}', {

   parameters: {gender = 'male'}

});

  

(Note the use of SC.Query.local's optional third parameter, a data hash.  You must use this hash to pass in parameters, but the structure is very flexible.  You can move the condition string in as well, or pass on the "parameters" sub-hash, merely defining your parameters by name -

 

// This is equivalent to the previous example. 

var localMenQuery = SC.Query.local(MyApp.Contact, {

   conditions: 'gender = {gender}',

   gender: 'male'

});

 

- in order to enable the passing of multiple conditions, parameters, et al.)

 

The ability to separately define your conditions and parameters is particularly useful when your parameter isn't a string or a number, but an object - such as a "toOne" record attribute defined in your model.  For example, to find all the Event records which have been assigned to the contact with ID #4, first find the contact record, then pass that record to the query as a parameter:

 

// find contact record.

var contactRecord = MyApp.store.find(MyApp.Contact, 4);

// find events assigned to that contact.

var localEventQuery = SC.Query.local(MyApp.Event, {

   conditions: 'assignedTo = {contact}',

   contact: contactRecord

}); 

 

Ordering 

 

The SC.Query class allows you to define a sort order on your queries, similar to the "ORDER BY" syntax in SQL.  To do so, pass a string of comma separated key names in with the query's hash:

 

 

// find all contacts, sorted by first name.

var localContactsQuery = SC.Query.local(MyApp.Contact, {

   orderBy: 'firstName'

}); 

 
Finally, you can also use the hash parameter to pass any additional properties your data source may need.  For example, you can pass a "url" property to a remote query, which will be ignored by the DataStore but will be faithfully passed to your data source for use:
 

var remoteContacts = SC.Query.remote(MyApp.Contact, { 

  url: '/contact/foo.json' 

});

 

Modifying Queries

 

Query objects returned from these builder functions are frozen by default.  This means you cannot make further modifications to the query.  Queries are generally frozen because the store maps record arrays with search results to query objects.  If you modify some conditions or something on a query, the store will not be able to properly update its record arrays.  

 

Instead, if you have a query and decide you need to modify it, the best way to change it is to copy the query, make your property changes, and freeze it again.  This will result in a new Query instance that the store can use instead of the old results:

 

var men = SC.Query.local(MyApp.Contact, {

  conditions: “gender = {gender}”,

  parameters: { gender: “male” },

  orderBy: “firstName”

});

 

var women = men.copy()

               .set(‘parameters’, { gender: “female” })

               .freeze();

 

Managing Your Queries

 

Since queries may contain literally any property you want to add to them, the DataStore makes only a very limited attempt to determine query equivalence.  That is, if you create a two query instances with the same search conditions, the DataStore will usually treat these two instances as separate searches.

 

From a performance standpoint, this means you need to generally manage query instances to ensure you do not create too many of them in your application.  For example, let’s say you wanted to write a computed property on a Group record that will find all Contacts with the group, you could write something like this:

 

MyApp.Group = SC.Record.extend({

  contacts: function() {

    var query = SC.Query.create(MyApp.Contact, “group = {group}”, { group: this });

    return this.get(‘store’).find(query);

 }.property().cacheable()

});

 

This might seem like nice neat code normally.  For any group you can just do group.get(‘contacts’) to find all the locally loaded contacts matching the group.  The problem with this design is that every time this property is recomputed, it will generate a new query object.  The store, in turn, will create a new record array and perform the search from scratch.

 

Instead, you should make sure you reuse a query object as often as you can.  For example, you could rewrite the above like this:

 

MyApp.Group = SC.Record.extend({

  contacts: function() {

    var query = this._query;

    if (!query) {

      query = this._query = SC.Query.create(MyApp.Contact, “group = {group}”, { group: this });

    }

 

    return this.get(‘store’).find(query);

 }.property().cacheable()

});

 

With this design you will compute a new query object once per group and then reuse those queries afterwards.  This will make recaching the contacts property on your group much faster and allow the store to optimize searches for you as well.

 

Even if you don’t take so much care to control how many queries you create, the DataStore framework will still work for you.  They key thing to keep in mind though is that if you can control your number of queries, you will be able to maximize your performance. 

 

Local Query Notes

 

Local queries search for results only on data that been loaded into your in-memory data store.  By restricting your search to only client-side data, the DataStore framework is able to handle some extra things for you.  Specifically:

 

  • The search will be performed for your automatically on demand.
  • As records are added/removed from the state or change state, the query results will be updated automatically to reflect any changes.
  • You can specify the actual search and order conditions using a simple query language called SproutCore Query Language or SCQL.

 

Local Queries and the Server

 

Note that even though a local query is restricted to in-memory data, this doesn’t necessarily mean that you can’t load data from the server as a result.  When you find() a local query for the first time, the store will always call a special method on your data source, asking it to fetch any related records from the server.  In response, your data source can always choose to load records from the server for the query.

 

Remote Query Notes

 

You can generally define a local query anyway you like using SCQL.  Remote queries, on the other hand, only work if you specifically implement support for them in your data source.  (See the Data Source section for more info).

 

Because of this behavior, it is common to define remote queries up front as constants on your application or data source.  Then both app code and the data source code can rely on the same query instance.  For example, you might define a remote query to return all contacts thus:

 

MyApp.CONTACTS_QUERY = SC.Query.remote(MyApp.Contact);

 

Later on in both your app code and in your data source code, you can refer back to this constant; ensuring that if your code is using the same instance.

 

Moving On

 

Learn all about the SproutCore Query Language » 

or back to DataStore Programming Guide Home »

 

 

Comments (5)

Dave said

at 10:02 pm on Nov 25, 2009

I reordered this section, moving sections above "Modifying" around to align with the "Anatomy of a Query" chart, which I think makes things flow much better. I added several examples, including querying toOne'd records, so all that should be checked over for accuracy.

This is now quite a long section. pbWorks suggests that pages this large be split... just sayin.

rubyphunk said

at 3:11 am on Dec 29, 2009

Looks like passing orderBy as Array (orderBy: ['firstName']) isn't working with the current SC implementation.. You need to pass the order param as string: orderBy: 'firstName'.

rubyphunk said

at 3:14 am on Dec 29, 2009

You can change the sorting direction by adding ASC or DESC to the "order"-field.: orderBy: 'firstName DESC, createdAt ASC'

Maurits Lamers said

at 2:03 am on Mar 15, 2010

It is interesting to know that when you are using conditions the condition field tends to be camelcased. So if you have a field in a record called blah_id, and you want to be able to query based on that value, put the condition in as blahId!

Maurits Lamers said

at 2:14 am on Mar 15, 2010

Please ignore this comment. Error on my side :)

You don't have permission to comment on this page.