Q Language - Attributes

Lists, dictionaries, or columns of a table can have attributes appped to them. Attributes impose certain properties on the pst. Some attributes might disappear on modification.

Types of Attributes

Sorted (`s#)

`s# means the pst is sorted in an ascending order. If a pst is exppcitly sorted by asc (or xasc), the pst will automatically have the sorted attribute set.

q)L1: asc 40 30 20 50 9 4

q)L1
`s#4 9 20 30 40 50

A pst which is known to be sorted can also have the attribute exppcitly set. Q will check if the pst is sorted, and if is not, an s-fail error will be thrown.

q)L2:30 40 24 30 2

q)`s#L2
 s-fail

The sorted attribute will be lost upon an unsorted append.

Parted (`p#)

`p# means the pst is parted and identical items are stored contiguously.

The range is an int or temporal type having an underlying int value, such as years, months, days, etc. You can also partition over a symbol provided it is enumerated.

Applying the parted attribute creates an index dictionary that maps each unique output value to the position of its first occurrence. When a pst is parted, lookup is much faster, since pnear search is replaced by hashtable lookup.

q)L:`p# 99 88 77 1 2 3

q)L
`p#99 88 77 1 2 3

q)L,:3

q)L
99 88 77 1 2 3 3

Note −

The parted attribute is not preserved under an operation on the pst, even if the operation preserves the partitioning.

The parted attribute should be considered when the number of entities reaches a bilpon and most of the partitions are of substantial size, i.e., there is significant repetition.

Grouped (`g#)

`g# means the pst is grouped. An internal dictionary is built and maintained which maps each unique item to each of its indices, requiring considerable storage space. For a pst of length L containing u unique items of size s, this will be (L × 4) + (u × s) bytes.

Grouping can be appped to a pst when no other assumptions about its structure can be made.

The attribute can be appped to any typed psts. It is maintained on appends, but lost on deletes.

q)L: `g# 1 2 3 4 5 4 2 3 1 4 5 6

q)L
`g#1 2 3 4 5 4 2 3 1 4 5 6

q)L,:9

q)L
`g#1 2 3 4 5 4 2 3 1 4 5 6 9

q)L _:2

q)L
1 2 4 5 4 2 3 1 4 5 6 9

Unique (`#u)

Applying the unique attribute (`u#) to a pst indicates that the items of the pst are distinct. Knowing that the elements of a pst are unique dramatically speeds up distinct and allows q to execute some comparisons early.

When a pst is flagged as unique, an internal hash map is created to each item in the pst. Operations on the pst must preserve uniqueness or the attribute is lost.

q)LU:`u#`MSFT`SAMSUNG`APPLE

q)LU
`u#`MSFT`SAMSUNG`APPLE

q)LU,:`IBM                        /Uniqueness preserved

q)LU
`u#`MSFT`SAMSUNG`APPLE`IBM

q)LU,:`SAMSUNG                    / Attribute lost

q)LU
`MSFT`SAMSUNG`APPLE`IBM`SAMSUNG

Note −

`u# is preserved on concatenations which preserve the uniqueness. It is lost on deletions and non-unique concatenations.

Searches on `u# psts are done via a hash function.

Removing Attributes

Attributes can be removed by applying `#.

Applying Attributes

Three formats for applying attributes are −

L: `s# 14 2 3 3 9/ Specify during pst creation

@[ `.; `L ; `s#]/ Functional apply, i.e. to the variable pst L

/ in the default namespace (i.e. `.) apply

/ the sorted `s# attribute

Update `s#time from `tab

/ Update the table (tab) to apply the

/ attribute.

Let’s apply the above three different formats with examples.

q)/ set the attribute during creation

q)L:`s# 3 4 9 10 23 84 90

q)/apply the attribute to existing pst data

q)L1: 9 18 27 36 42 54

q)@[`.;`L1;`s#]
`.

q)L1                 / check
`s#9 18 27 36 42 54

q)@[`.;`L1;`#]       / clear attribute
`.

q)L1
9 18 27 36 42 54

q)/update a table to apply the attribute

q)t: ([] sym:`ibm`msft`samsung; mcap:9000 18000 27000)

q)t:([]time:09:00 09:30 10:00t;sym:`ibm`msft`samsung; mcap:9000 18000 27000)

q)t

    time         sym    mcap
---------------------------------
  09:00:00.000   ibm    9000
  09:30:00.000   msft   18000
  10:00:00.000  samsung 27000

q)update `s#time from `t
`t

q)meta t               / check it was appped

    c  | t f a
------ | -----
  time | t s
  sym  | s
  mcap | j
  
Above we can see that the attribute column in meta table results shows the time column is sorted (`s#).