Code Search

Deveo supports configuring Elasticsearch (5.x) for searching code in Git repositories. Elasticsearch is not included as part of Deveo packages. It can be installed in multiple ways depending on the needs: inside a Deveo instance (only recommended for small instances), as a separate machine, or as a cluster of separate machines. See the official documentation for installing Elasticsearch and for configuring production deployment.

Prerequisites

First step is to estimate the index size based on the storage size of Git repositories. Using this figure with future growth in mind, you can decide the type of Elasticsearch installation and number of shards to create. See capacity planning for more details. Note that changing the number of shards requires recreating the index and reindexing all the repositories.

Configuration

Once you have a running instance of Elasticseach (5.x), you can configure Deveo to use it.

Combo

Append the following minimum configuration to /var/opt/deveo/shared/deveo.json and make sure to add the settings under the existing backend key.

{
  "backend": {
    "search_engine": "elasticsearch",
    "es_hosts": [
        {
            "host": "your.elastic.host",
            "port": 9200
        }
    ]
  }
}

See backend section in configuration flags for additional settings:

{
  "backend": {
    "search_engine": "elasticsearch",
    "es_index_prefix": "optional",
    "es_number_of_shards": 5,
    "es_number_of_replicas": 1,
    "es_hosts": [
        {
            "host": "your.elastic.host",
            "port": 9200,
            "user": "username",
            "password": "password",
            "scheme": "https"
        }
    ]
  }
}

Apply the changes by reconfiguring Deveo:

sudo deveo-ctl reconfigure

Create the Elasticsearch index as a deveo user. Make sure to load a new session for deveo user after running reconfigure:

sudo su - deveo
cd /opt/deveo/application/backend/current/
bundle exec rake deveo:search:create_index

Cluster and HA

Follow the same steps as in Combo section above in one of the web nodes. After creating the index, reconfigure rest of the web nodes before enabling search in Deveo Client.

sudo deveo-ctl reconfigure

After configuration, the search can be enabled inside a company. Login to Deveo Client as a company admin and select company features from the company overview. Enable "Code Search" under the features and wait for the indexing to complete after saving. The initial indexing can take a long time depending on the size of the repositories. You can see the indexing status in the company overview.

Index management

After changing index configurations later on, such as number shards, the index can be recreated as a deveo user. Make sure to load a new session for deveo user after running reconfigure:

sudo su - deveo
cd /opt/deveo/application/backend/current/

Create, delete, or recreate the index:

bundle exec rake deveo:search:create_index

bundle exec rake deveo:search:delete_index

bundle exec rake deveo:search:recreate_index

Refresh all repositories for a company (by short_name) since last indexing, or perform full reindexing (can take a long time):

bundle exec rake deveo:search:reindex_company[company]

bundle exec rake deveo:search:reindex_company[company,full]

Refresh a repository (by short_names) since last indexing, or perform full reindexing:

bundle exec rake deveo:search:reindex_repository[company,project,repository]

bundle exec rake deveo:search:reindex_repository[company,project,repository,full]
Updated on: 27 July 2017