PagedResult as a list of loaded data
Description and example of the implementation of the PagedResult output data, also called a paged result. The described format of the transmitted data is applicable wherever the response to the request returns lists of very large volume, which the client is unable to process in a single operation, where it is necessary to divide these lists into pages. PagedResult is a kind of response standard used in communication with the HgDB system. This chapter describes the type structure in the form of JSON (for REST services) and XML (for SOAP services). The object also has its representation (implementation) in Java and is used there in RMI communication.
PagedResult​
| Layers in which used | Business |
| Type | Business object |
| Java interface | pro.ibpm.mercury.logic.paging.IPagedResult<E,P extends IPage>, IMrcObject, IMrcPropertyAttrs |
| Java implementation | pro.ibpm.mercury.dto.paging.PagedResult<E> |
| DTO implementation | pro.ibpm.mercury.dto.paging.PagingInfoTransportable |
| XML definition | hgdb-mrc-object-3.0.xsd |
Paged result. An object representing the result of the response to a request sent to services presenting lists and implementing case search mechanisms in the HgDB system. The object definition is parameterized (see Java implementation), where the E parameter is the definition of the list element it represents. The object interface also contains the P parameter, which represents the definition of the page object, but in all implementations used in HgDB, it is represented by a single definition described in this article.
The PagedResult object has many implementations, which are used in the logical and business layers. Since it is a representation of a list of data, we can distinguish two main implementations:
- the
MrcPagedResultimplementation storing objects of typeMrcObjectin the list (see Case as a universal MRC object), - the implementation storing any objects representing cases in the list (see Case as any object)
Constant parameters​
PagedResult occurs in the implementation as a universal MrcObject object (see Case as a universal MRC object) named MrcPagedResult. In its case, there is no possibility to add your own parameters, because it is not an object predefined by the system, unlike the definition of case objects. The values ​​of the parameters are constant and defined in the implementation.
<propertyNames>
<propertyName>resultSize</propertyName>
<propertyName>result</propertyName>
<propertyName>message</propertyName>
<propertyName>executionTime</propertyName>
<propertyName>currentPageInfo</propertyName>
<propertyName>firstPageInfo</propertyName>
<propertyName>previousPageInfo</propertyName>
<propertyName>nextPageInfo</propertyName>
<propertyName>lastPageInfo</propertyName>
<propertyName>allPages</propertyName>
<propertyName>pagingParams</propertyName>
</propertyNames>
resultSize​
| Parameter name | resultSize |
| Type | Long |
A parameter defining the number of all found elements. If no elements were found, the value of this parameter is 0.
If the request context (see the definition of the Context object is set to the maxResults field, then we must be aware that if there are more elements in the system, the resultSize will take the value defined in this field.
Use case: When implementing a control panel in a result presentation table on web pages, we can use this value to provide information about the full size of the result:
result​
| Parameter name | result |
| Type | List<ANY>, MrcList |
A list of elements appearing on a given result page. The definition of the element depends on the service called (remember that the object is parameterized). You should find out how the list element is defined in the description of the service/method you used to get it.
The length of the list depends on the declared page size defined in the request sent to the service.
Below is an example of a list definition in JSON as an implementation of an ANY list of cases without headers1:
[
{
"status": "Active",
"name": "aatest",
"priv": "RO",
"users": [
"slawa1233",
"slawas2",
"slawas3",
"slawas4",
"ttest2345"
]
},
{
"status": "Active",
"name": "wtest",
"priv": "RW",
"users": [
"slawa1233",
"slawas2",
"slawas3",
"slawas4",
"ttest2345"
]
}
]
In the case of implementing a list of MrcObject objects (see Case as a universal MRC object), the list will contain MrcObject objects, which are the representation of cases in the HgDB system.
message​
| Parameter name | message |
| Type | String |
Message related to the result. It can take the following values:
FRAGMENT- this means that the list of elements contained in the result field is only a fragment of the entire result. A value characteristic for the result page, which is not the last page (there are subsequent pages of the given result).ALL- this means that all elements of the result have already been downloaded. A value characteristic for the result page, which is the last page (there are no subsequent pages of the given result or it is the last page of the downloaded result).NO_DATA_FOUND- this means that the result is empty, no data was found.
executionTime​
| Parameter name | executionTime |
| Type | Long |
Information field, time of request execution by the service, specified in milliseconds.
currentPageInfo​
| Parameter name | currentPageInfo |
| Type | Page, MrcPage |
Data about the definition of the current result page. Contains the page number and its size.
{
"size": 20,
"number": 2
}
Use case: When implementing a control panel in a result presentation table on web pages, we can use this value to provide information about which result page we are on:
firstPageInfo​
| Parameter name | firstPageInfo |
| Type | Page, MrcPage |
Data about the first result page. Contains the page number and its size.
{
"size": 20,
"number": 1
}
Use case: When implementing a control panel in a result presentation table on web pages, we can use this value to create a link to go to the first page of the result.
previousPageInfo​
| Parameter name | previousPageInfo |
| Type | Page, MrcPage |
Data about the previous page of the result. Contains the page number and its size.
{
"size": 20,
"number": 1
}
Use case: When implementing a control panel in a result presentation table on web pages, we can use this value to create a link to the previous page:
nextPageInfo​
| Parameter name | nextPageInfo |
| Type | Page, MrcPage |
Data about the next page of the result. Contains the page number and its size.
{
"size": 20,
"number": 3
}
Use case: When implementing a control panel in a result presentation table on web pages, we can use this value to create a link to the next page:
lastPageInfo​
| Parameter name | lastPageInfo |
| Type | Page, MrcPage |
Data about the last page of the result. Contains the page number and its size.
{
"size": 20,
"number": 246
}
Use case: When implementing a control panel in a result presentation table on web pages, we can use this value to create a link to the last page of the result.
allPages​
| Parameter name | allPages |
| Type | List<Page>, List<MrcPage> |
A list of definitions of up to 10 pages occurring near the current page.
[
{
"size": 20,
"number": 1
},
{
"size": 20,
"number": 2
},
{
"size": 20,
"number": 3
},
//...
]
Use case: When implementing a control panel in a result presentation table on web pages, we can use this list to create links to the nearest pages.
pagingParams​
| Parameter name | pagingParams |
| Type | PagingParams, MrcPagingParams |
Definition of paging, parameters for building pages of the submitted result. Auxiliary data, presenting the rules for creating a paged result.
{
"offset": 0,
"cursorOfPage": 20,
"maxPageSize": 20,
"maxCount": 10000,
"pageSize": 20,
"page": {
"size": 20,
"number": 1
},
"valid": "true",
"isReadOnly": "false"
}
Downloading the full result​
In backend solutions, we can very often encounter the problem of downloading the entire result, e.g. when performing a data export operation, data transfer between systems. Below is an example algorithm for downloading all results. Of course, the implementation of downloading the entire result itself can be distributed across many threads, it all depends on the business requirements of the project being implemented and the skills of the programmers implementing it.
Data download algorithm​
The presented algorithm shows how useful it is to use the PagedResult object. Especially when processing (sending) a large volume of data obtained from HgDB services. The operation can be interrupted and continued at any point in the result download.
Description of the individual steps of the algorithm:
 |
|
Multithreaded download​
Based on the data stored in the resultSize and pagingParams.pageSize fields, we can define the number of parallel threads and divide the work to shorten the data download time, e.g. (very simple example) for the value:
resultSize= 120pagingParams.pageSize= 20
We will run 3 threads that will download data in parallel:
- thread one downloads pages 1 and 2
- thread two downloads pages 3 and 4
- thread three downloads pages 5 and 6
This will theoretically shorten the download time of the entire result by three times.
Download time estimation​
Additionally, we can use the value of the executionTime and lastPageInfo.pageSize fields to estimate the download time of the entire result. In the example, where:
executionTime= 643[ms]lastPageInfo.number= 246
We can estimate the operation completion time:
- for downloading in one thread, the estimated completion time is
executionTime * lastPafeInfo.number=>643[ms] * 246 = 158178[ms]=~ 158[s]=~ 2.5[min]. - for downloading in five threads, the estimated operation completion time can be reduced to
30[s].
Of course, we can write the download mechanism in such a way that the estimated time can be modified with the incoming data (data pages), which will update the information about the execution time of the operation on a single page, and the number of pages remaining to be downloaded. It should be emphasized that the time of downloading the first page of the result also includes the time of executing the appropriate query, and the data of the remaining pages can be stored in the system cache - it also depends on the HgDB configuration, namely how long the result expiration time in the cache is defined - the default value is 5 minutes.
ExcelData​
| Layers in which used | Business |
| Type | Business object, external |
| Java interface | org.iron.poi.excel.core.api.IExcelData |
| Java implementation | org.iron.poi.excel.core.beans.ExcelData |
| DTO implementation | org.iron.poi.excel.core.beans.ExcelData |
ExcelData is a special implementation of a paged search result (PagedResult). This type is a kind of communication standard between individual components of the HgDB system and is used as a universal data carrier in the form of relations (arrays).
This standard was created as part of the Iron - POI Excel Server project to present data stored in Excel files - hence its name. The Java implementation of the object is located in the Iron product implementation library.
In this implementation, the result field list element is a value map, the key of which is the column name consistent with the naming used in the Excel spreadsheet. Additionally, in order to identify the result row, an item called rowIndx was added, which represents the unique key of the transferred data and contains the row number of the returned result.
{
"A": "2019-01-02 13",
"B": "0",
"C": "4",
"rowIndx": "60"
}
Since the list item does not contain all the data that would allow for a full description of the relationship, the following fields have been added to the ExcelData implementation.
header​
| Parameter name | header |
| Type | Map |
Header, additional data representing the names of individual columns. This is a map whose key is the name of the Excel column, and the value is a customized name of that column.
{
"A": "Date",
"B": "mrc_createDate",
"C": "mrc_endDate"
}
columnTypes​
| Parameter name | columnTypes |
| Type | Map |
Column type definitions. This is a map whose key is the name of an Excel column and the value is the type of that column. Below is a list of possible column type names:
STRING- represents the Java object typejava.lang.StringINTEGER- represents the Java object typejava.lang.IntegerINT- represents the Java primitive typeintLONG- represents the Java object typejava.lang.LongSHORT- represents the Java object typejava.lang.ShortFLOAT- represents the Java object typejava.lang.FloatDOUBLE- represents the Java object typejava.lang.DoubleBIGDECIMAL- represents the Java object typejava.lang.BigDecimalBYTE- represents the Java object typejava.lang.ByteBOOLEAN- represents the Java object typejava.lang.BooleanDATE- represents the Java object typejava.lang.DateCALENDAR- represents the Java object typejava.lang.Calendar
{
"A": "Date",
"B": "mrc_createDate",
"C": "mrc_endDate"
}
Example of presenting an ExcelData object​
Example of presenting an ExcelData object in JSON
{
"result": [
{
"A": "2019-02-01 10",
"B": "1",
"C": "0",
"rowIndx": "1"
},
{
"A": "2019-02-01 09",
"B": "6",
"C": "0",
"rowIndx": "2"
},
{
"A": "2019-01-25 13",
"B": "22796",
"C": "0",
"rowIndx": "3"
},
{
"A": "2019-01-24 21",
"B": "649",
"C": "0",
"rowIndx": "4"
},
{
"A": "2019-01-23 08",
"B": "7",
"C": "0",
"rowIndx": "5"
},
{
"A": "2019-01-21 18",
"B": "1",
"C": "0",
"rowIndx": "6"
},
{
"A": "2019-01-21 15",
"B": "11",
"C": "0",
"rowIndx": "7"
},
{
"A": "2019-01-12 04",
"B": "1",
"C": "0",
"rowIndx": "8"
},
{
"A": "2019-01-12 03",
"B": "23809",
"C": "0",
"rowIndx": "9"
},
{
"A": "2019-01-10 15",
"B": "0",
"C": "4",
"rowIndx": "10"
},
{
"A": "2019-01-10 14",
"B": "0",
"C": "2",
"rowIndx": "11"
},
{
"A": "2019-01-10 13",
"B": "0",
"C": "2",
"rowIndx": "12"
},
{
"A": "2019-01-10 12",
"B": "0",
"C": "4",
"rowIndx": "13"
},
{
"A": "2019-01-10 11",
"B": "0",
"C": "2",
"rowIndx": "14"
},
{
"A": "2019-01-10 10",
"B": "0",
"C": "2",
"rowIndx": "15"
},
{
"A": "2019-01-10 07",
"B": "0",
"C": "2",
"rowIndx": "16"
},
{
"A": "2019-01-10 05",
"B": "0",
"C": "6",
"rowIndx": "17"
},
{
"A": "2019-01-09 20",
"B": "0",
"C": "2",
"rowIndx": "18"
},
{
"A": "2019-01-09 16",
"B": "0",
"C": "2",
"rowIndx": "19"
},
{
"A": "2019-01-09 15",
"B": "0",
"C": "4",
"rowIndx": "20"
}
],
"message": "FRAGMENT",
"resultSize": 134,
"currentPageInfo": {
"size": 20,
"number": 1
},
"firstPageInfo": {
"size": 20,
"number": 1
},
"previousPageInfo": {
"size": 20,
"number": 1
},
"nextPageInfo": {
"size": 20,
"number": 2
},
"lastPageInfo": {
"size": 40,
"number": 7
},
"header": {
"A": "Date",
"B": "mrc_createDate",
"C": "mrc_endDate"
},
"columnTypes": {
"A": "STRING",
"B": "LONG",
"C": "LONG"
}
}
QueryStats​
| Layers used | Business |
| Type | Business object |
| Java interface | - |
| Java implementation | org.mercury.lucene.query.QueryStats#QueryStatsCollector |
| DTO implementation | org.mercury.lucene.query.QueryStats#QueryStatsCollector |
QueryStats - detailed statistics of query execution. The object is attached (not always - see the details of the description of the method response of a given service) to the response of the service performing the search and aggregation action in the Lucene index. Analysis of its value can help optimize queries performed by the application. The object has the following fields:
| Field | Description | Type | Example |
|---|---|---|---|
| allExecTime | Query execution time expressed in milliseconds. | Long | 1339 |
| mainPrepareTime | Time spent preparing the query (parsing, preparing the execution plan, etc.) expressed in milliseconds. | Long | 11 |
| collValidationTime | Time spent validating the collector data (sum of all validations) - time expressed in milliseconds. | Long | 1 |
| mainCollExecTime | Main collector execution time (sum of all calls) - time expressed in milliseconds. | Long | 0 |
| mainCollExecCount | Number of calls to the main collector (number of processed documents, verification of search criteria). | Long | 47281 |
| subCollExecTime | Subordinate collector execution time (sum of all calls) - time expressed in milliseconds. | Long | 4 |
| subCollExecCount | Number of dependent collector calls (number of processed documents, verification of search criteria). | Long | 8 |
| subExecLeve1Time | Query execution time at level 1 (for queries with subqueries, criteria for complex case fields) - time expressed in milliseconds. | Long | 1337 |
| subExecLeve2Time | Query execution time at level 2 (for queries with subqueries, criteria for complex case fields and data aggregation) - time expressed in milliseconds. | Long | 0 |
| subExecLeve3Time | Query execution time at level 3 (for queries with subqueries, criteria for complex case fields and data aggregation) - time expressed in milliseconds. | Long | 0 |
| subExecLeve4Time | Query execution time at level 4 (for queries with subqueries, criteria for complex case fields and data aggregation) - time expressed in milliseconds. | Long | 0 |
| initResultSetTime | Time spent preparing the result expressed in milliseconds. | Long | 6 |
| resultSetSize | Total result size. | Long | 3 |
{
"allExecTime": 1339,
"mainPrepareTime": 11,
"collValidationTime": 1,
"mainCollExecTime": 0,
"mainCollExecCount": 47281,
"subCollExecTime": 4,
"subCollExecCount": 8,
"subExecLeve1Time": 1337,
"subExecLeve2Time": 0,
"subExecLeve3Time": 0,
"subExecLeve4Time": 0,
"initResultSetTime": 6,
"resultSetSize": 3
}
Footnotes​
-
In response to a request to get a list of cases, the result may not contain case headers. Whether the case header should be present in the result is determined by the
ignoreCaseHeaderInResponseparameter (ignore case header in response) passed in the context object in the request to the service. This parameter takes the boolean values ​​trueorfalse. The default value of this parameter isfalse, so if it is not set, the case header is returned. ↩