SwiftStack Client Metadata Search

Once you have set up Metadata Search in your SwiftStack cluster, you can use SwiftStack Client to search for objects.

When using the client to search for objects, search queries are powered by Elasticsearch, and storage operations make use of the Swift API.

Setting up Metadata Search

To allow SwiftStack Client to search for objects in your cluster, follow these steps:

1. Open the client, and create an Account if you don't already have one.

2. Navigate to the Metadata Search tab on the left.

   

3. Click Add a Search.

   

4. Choose a SwiftStack Account from the credentials dropdown. The Swift user credentials stored with this account will be used for Swift operations.

   

   Note

   You can search for objects in any account you like, but you will not be able to download or modify them unless this user has permission to do so.

5. Enter the URL to your Elasticsearch cluster to the Elasticsearch URL text field, including the index your metadata is stored in.

   The URL and index should be the same as those used when setting up a Metadata Search Mapping in the SwiftStack Controller (see SwiftStack Metadata Search).

   Include your username and password for Elasticsearch, if any, in the URL. If you do not do this, and if authentication is needed, SwiftStack client will attempt to reuse the credentials stored with the Swift account you chose above.

6. If you would like to restrict searches to a single account (recommended) or container, check the appropriate control in Search Constraints and provide the account or container name.

   

   Tip

   We recommend you constrain searches to at least an account where possible, to ease the performance impact on Elasticsearch.

7. Add a search term to the Query bar. Below, we're searching for any objects whose metadata contains the word "expectations", within the account AUTH_gutenberg.

   

8. Click Run Search. The client will display matching objects.

The Metadata Search control will indicate that a search is active, and you will only be shown objects matching your current search.

You can open the Metadata Search dialog to edit your search at any time by clicking the Metadata Search control, or by pressing Control-F (Windows) or Alt-Space (Mac and Linux).

Saving Searches

You can save any search from the Metadata Search dialog.

1. Open the Metadata Search Dialog.
2. Click Save As... to name and save the search.

Saved searches are shown in the Metadata Search tab. Click Run Search to search again, or use the settings control to see more options.

Automatic Search

Rather than setting up a new search and adding Elasticsearch details each time, you can also tell SwiftStack Client to automatically use an Elasticsearch URL for any container, as long as it has a Metadata Search Mapping.

To do this:

1. Open your account, and navigate to the container you'd like to search.

2. Click Container Properties.

   

3. Add a metadata key named com-swiftstack-mdsync-elasticsearch-uri, setting its value to the URL of your Elasticsearch cluster, including the index, and a username or password, if needed.

   

4. Click Update Metadata, and close the modal.

5. Click Refresh. If the client can connect to the Elasticsearch URL, it will indicate that metadata search is enabled.

From now on, you will be able to perform searches within this container directly by using the Metadata Search control (or by pressing Control-F on windows / Alt-Space on a Mac or Linux machine).

To enable search in this way for an entire account, set the com-swiftstack-mdsync-elasticsearch-uri metadata key at the account level.

Using Metadata Search

Search Types

SwiftStack Client provides two tools for search: the Search Editor and the Query Bar.

Search Editor

For many people, the easiest way to create powerful searches will be to use the Search Editor - a graphical tool that helps you to construct metadata searches.

To use the query editor, select Editor using the Search Type switch in the Metadata Search dialog.

The query editor allows you to build a search using Conditions. For a search to return objects, all of the conditions you add must match.

To add a condition:

1. Click Add.

2. Click Choose Fields to pick one or more metadata fields for the condition. To see the data type of each field, hover your mouse over the field name.
3. Specify your criteria for a match, for these field(s), in the Condition column. The controls shown will change depending on the type of the fields you have chosen:

• Text fields will provide a text box with different matching options.
• Numeric and date fields will allow you to search for a single value, or for a range of values.
• Boolean fields will allow you to choose if values should be true or false.
• When a field is an array (a list of items), objects will be shown if any value in the array matches your condition.
• When a field is a range (a numeric or date range), you can decide whether matches should lie within the range, lie outside of it, or intersect it.
• Other field types will show you a text query box. Use the syntax described in Advanced Queries to search these types.

4. Select a Type for the condition. Condition types are:

• Must Match: The fields you select must match your criteria for object to be shown. In boolean logic, this is equivalent to AND.
• Must Not Match: The fields you select must not match your criteria for objects to be shown. This is equivalent to boolean NOT.
• Match Any One: At least one condition with the Match Any One type must result in a match for objects to be shown. This is equivalent to boolean OR.

5. Repeat steps 1 to 5 to create as many conditions as you like in order to find your objects.
6. Click Run Search to search for objects.

Query Bar

The query bar is the default search tool in SwiftStack client. It allows you to search object metadata using text.

To use the query bar, select Query using the Search Type switch in the Metadata Search dialog.

The dropdown to the left of the query bar allows you to select a query type:

Simple Queries

Simple queries match against all object metadata fields, using a "search engine" like syntax. Enter the text you're interested in into the search bar.

• Objects whose metadata matches any of the words you enter will be returned.
• To match a group of words as a phrase, wrap them in double quotes.

You can also use the special characters below:

• + indicates that the word must match (boolean AND).
• - indicates that the word must not match (boolean NOT).
• | indicates that the word may match (boolean OR).
• * matches any character or characters.
• ~ after a word indicates fuzziness (the number of characters that would need to be changed to ensure a match).
• ( and ) allow you to group sections of your query together.

To search for any of these special characters, you should escape them with \.

Examples

Search for metadata containing the word "expectations":

expectations

Search for metadata starting with "expect":

expect*

Search for metadata containing the phrase "the best of times":

"the best of times"

Search for metadata that contains "best" or does not include "worst":

+best -worst

Search for metadata that contains "best" and does not include "worst":

+best +-worst

Search for metadata that contains "Pirrip", allowing up to two spelling mistakes:

Pillip~2

Search for metadata that either matches both "married the blacksmith" and "pip", or matches both "white rabbit" and "oh dear, oh dear":

(+"married the blacksmith" + "pip") | ( +"white rabbit" +"oh dear oh dear")

For more details on Simple Search syntax, see the Elasticsearch documentation.

Advanced Queries

Advanced queries provide a more expressive search syntax that allows you to specify the metadata fields you'd like to search for.

As with Simple Queries, any terms that match will return objects, unless you specify otherwise using boolean operators. To match a phrase, enclose it in double quotes.

Advanced queries allow you to use the following search features:

• Field Selection: Specify the field you'd like to match against by prefixing your text with the field name, followed by :.
• Boolean Logic is supported via the + and - special characters, by the AND, OR, or NOT keywords, or by &&, ||, and !.
• Group terms with ( and ).
• Wildcards: * matches any number of any characters. ? matches any single character.
• Fuzziness, described above, is supported via the ~ operator for single words.
• Word proximity is also supported via the ~ operator when used after a phrase. This allows you to specify the number of whole-word edits allowed in order to make the metadata match.
• Ranges for dates and numbers are supported. See the examples below.
• Regular expressions are also supported, though they operate on single words only - not whole phrases.

Note

The behaviour of boolean operators in advanced queries differs from that of simple queries.

For more details of boolean operators under Advanced Search, see https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-query-string-query.html#_boolean_operators.

Examples

Match objects where the title metadata field matches "great":

title:great

Match metadata containing the phrase "the best of times" and the word "worst":

"best of times" AND worst

Match metadata where author is either "charles" or "lewis" and title matches "great" or "wonderland":

(author:charles OR author:lewis) AND (title:great OR title:wonderland)

Use wildcards to match the word "expectations":

*pectation?

Use fuzziness to match the word "expectations", allowing for up to one spelling mistake:

expectationz~1

Use word proximity to match either "quick fox" or "fox quick":

"fox quick"~1

Search for a date between midnight, 31 Jan 1999 and noon, 31 Jan 2001:

[1999-01-31 00:00:00 TO 2001-01-31 12:00:00]

Search for a number between 13.2 and 42:

[13.2 TO 42]

For more details on Advanced Search syntax in general, see the Elasticsearch query string documentation.