Automate code search configuration

ServiceNow / By /

A couple of years ago, I was helping a client with the migration to new ServiceNow instances.

One important part of the process was cleaning up and doing some code refactoring. But finding all the usages of certain fields and making sure we didn’t miss anything important was very time-consuming.

I am a big fan of the /code search feature from SN Utils, and knew it was possible to add more tables and fields. But doing that manually still felt exhausting.

Could we automate it?

Would running the /code search command for hundreds of fields break the instance or take ages to execute?

Keep reading to find out, or click here to download the update set with the code.

Automatically update tables and fields for code search

The solution was to create a scheduled job that runs daily and updates which tables and fields are included in the code search.

Key points:

  • The job populates the Code Search Tables [sn_codesearch_table] table with searchable fields.
  • Configurable via system properties
  • Filters exclude problematic scopes and tables, avoiding permission errors1.
  • Newly installed plugins get tracked automatically without manual intervention.
  • New custom tables or fields get tracked automatically without manual intervention.

Code and Dynamic Configuration

System property

To facilitate the configuration, a system property contains a JSON that defines some parameters:

  • searchGroup: The sys_id of the search group this configuration applies to.
  • fieldTypes: An exact-match list of internal field types to include in the search.
  • fieldTypeLabels: Any field whose label contains one of these strings will also be included.
  • excludeScopes: A list of scope sys_ids to exclude from the search. This prevents errors and permission issues by skipping restricted or problematic scopes.
  • excludeTables: A list of specific table names to exclude. The default values filter out tables known to cause errors in vanilla Personal Developer Instances (PDIs) as of the Zurich release.
  • searchNames: A boolean indicating whether to include the “name” field in the search. [Warning: Tables that only have the “name” field and no other searchable fields will be skipped to avoid irrelevant entries]
{
  "searchGroup": "c2f8a6a2d7120200b6bddb0c825203cd",
  "fieldTypes": [
    "xml",
    "condition_string",
    "condition",
    "html_template",
    "css"
  ],
  "fieldTypeLabels": [
    "script"
  ],
  "excludeScopes": [],
  "excludeTables": [
    "sn_templated_snip_note_template",
    "sys_declarative_action_assignment",
    "sys_ux_sitemap_definition"
  ],
  "searchNames": true
}
Code language: JSON / JSON with Comments (json)

Scheduled job

The code starts by loading a JSON configuration property that defines parameters such as which scopes, tables, and field types to include or exclude for the automated code search.

It then builds a dynamic query to retrieve all dictionary entries matching those criteria.

As it iterates over the results, it groups fields by their tables, skipping tables that only have the “name” field.

For each table collected, the code checks if there is already an entry in sn_codesearch_table for the configured search group.

If the existing search fields differ from the newly found ones, it updates the record; if no record exists, it creates one.

(function() {

    var config = JSON.parse(gs.getProperty('sn_codesearch.custom.autopopulate.config'));

    var query = 'sys_class_name=sys_dictionary' + '^' +
        buildQueryPart('excludeScopes', 'sys_scope!=', '^') +
        buildQueryPart('excludeTables', 'name!=', '^') +
        buildQueryPart('fieldTypes', 'internal_type=', '^OR') +
        buildQueryPart('fieldTypeLabels', 'internal_type.labelLIKE', '^OR') +
        (config.searchNames ? 'element=name' : '');

    var dictionaryGR = new GlideRecord('sys_dictionary');
    dictionaryGR.addEncodedQuery(query);
    dictionaryGR.query();

    var dictionary = []; // This structure simplifies further processing by organising fields per table

    while (dictionaryGR.next()) {
        var tableName = dictionaryGR.name.toString();
        var fieldName = dictionaryGR.element.toString();

        if (!dictionary[tableName]) {
            dictionary[tableName] = [];
        }

        dictionary[tableName].push(fieldName);
    }

    for (var i in dictionary) {
        var newFields = dictionary[i];
        //Skip if only "name" field was found
        if (newFields.length == 1 && newFields[0] == 'name') {
            continue;
        }
        var codeSearchGR = new GlideRecord('sn_codesearch_table');
        codeSearchGR.addQuery('table', i);
        codeSearchGR.addQuery('search_group', config.searchGroup);

        codeSearchGR.query();
        if (codeSearchGR.next()) {
            //If new list of fields is different from existing one, update record
            var existingFields = codeSearchGR.getValue('search_fields') ? codeSearchGR.getValue('search_fields').split(',') : [];
            var arrayUtil = new ArrayUtil();

            if (arrayUtil.diff(existingFields, newFields).length > 0) {
                codeSearchGR.setValue('search_fields', newFields.join(','));
                codeSearchGR.update();
            }
        } else {
            //Create new record
            codeSearchGR.setValue('table', i);
            codeSearchGR.setValue('search_group', config.searchGroup);
            codeSearchGR.setValue('search_fields', newFields.join(','));
            codeSearchGR.insert();

        }
    }

    function buildQueryPart(configProperty, text, separator) {
        return config[configProperty] ? config[configProperty].map(item => text + item).join(separator) + separator : '';
    }
})();

Code language: JavaScript (javascript)

Other considerations

  • For tables such as sys_properties, that might contain relevant data in a “string” field, you can add search records manually.
  • The search is faster than I expected, even when you search hundreds of tables. But you might want to use multiple search groups to keep a basic code search plus a comprehensive code search.

Why don’t you publish it to ServiceNow Share?

I planned to contribute it there alongside other utilities we use at SwissFlowIt, but I found that Mark Roethof has already shared a similar solution.

It’s great to see others independently arriving at the same idea.

You might also want to check out Maik Skoddow’s implementation on GitHub for another version.

Download “Automate code search configuration” update set

  1. This was added to avoid errors being displayed when trying to search on restricted tables, which is quite annoying:
    – Read operation on table ‘xxxx’ from scope ‘xxxx’ was denied. The application ‘xxxx’ must declare a Restricted Caller Access request
    – Read operation against ‘xxxx’ from scope ‘xxxx’ has been refused due to the table’s cross-scope access policy ↩︎

Related Posts