Quickstart: Build a API for Table app with Java SDK and Azure Cosmos DB

APPLIES TO: Table

This quickstart shows how to access the Azure Cosmos DB Tables API from a Java application. The Azure Cosmos DB Tables API is a schemaless data store allowing applications to store structured NoSQL data in the cloud. Because data is stored in a schemaless design, new properties (columns) are automatically added to the table when an object with a new attribute is added to the table.

Java applications can access the Azure Cosmos DB Tables API using the azure-data-tables client library.

Prerequisites

The sample application is written in Spring Boot 2.6.4, You can use either Visual Studio Code, or IntelliJ IDEA as an IDE.

If you don't have an Azure trial subscription, create a trial subscription before you begin.

Sample application

The sample application for this tutorial may be cloned or downloaded from the repository https://github.com/Azure-Samples/msdocs-azure-data-tables-sdk-java. Both a starter and completed app are included in the sample repository.

git clone https://github.com/Azure-Samples/msdocs-azure-data-tables-sdk-java

The sample application uses weather data as an example to demonstrate the capabilities of the Tables API. Objects representing weather observations are stored and retrieved using the API for Table, including storing objects with additional properties to demonstrate the schemaless capabilities of the Tables API.

1 - Create an Azure Cosmos DB account

You first need to create an Azure Cosmos DB Tables API account that will contain the table(s) used in your application. This can be done using the Azure portal, Azure CLI, or Azure PowerShell.

2 - Create a table

Next, you need to create a table within your Azure Cosmos DB account for your application to use. Unlike a traditional database, you only need to specify the name of the table, not the properties (columns) in the table. As data is loaded into your table, the properties (columns) will be automatically created as needed.

3 - Get Azure Cosmos DB connection string

To access your table(s) in Azure Cosmos DB, your app will need the table connection string for the CosmosDB Storage account. The connection string can be retrieved using the Azure portal, Azure CLI or Azure PowerShell.

The connection string for your Azure Cosmos DB account is considered an app secret and must be protected like any other app secret or password. This example uses the POM to store the connection string during development and make it available to the application.

<profiles>
    <profile>
        <id>local</id>
        <properties>
            <azure.tables.connection.string>
                <![CDATA[YOUR-DATA-TABLES-SERVICE-CONNECTION-STRING]]>
            </azure.tables.connection.string>
            <azure.tables.tableName>WeatherData</azure.tables.tableName>
        </properties>
        <activation>
            <activeByDefault>true</activeByDefault>
        </activation>
    </profile>
</profiles>

4 - Include the azure-data-tables package

To access the Azure Cosmos DB Tables API from a Java application, include the azure-data-tables package.

<dependency>
    <groupId>com.azure</groupId>
    <artifactId>azure-data-tables</artifactId>
    <version>12.2.1</version>
</dependency>

---

5 - Configure the Table client in TableServiceConfig.java

The Azure SDK communicates with Azure using client objects to execute different operations against Azure. The TableClient object is the object used to communicate with the Azure Cosmos DB Tables API.

An application will typically create a single TableClient object per table to be used throughout the application. It's recommended to indicate that a method produces a TableClient object bean to be managed by the Spring container and as a singleton to accomplish this.

In the TableServiceConfig.java file of the application, edit the tableClientConfiguration() method to match the following code snippet:

@Configuration
public class TableServiceConfiguration {

    private static String TABLE_NAME;

    private static String CONNECTION_STRING;

    @Value("${azure.tables.connection.string}")
    public void setConnectionStringStatic(String connectionString) {
        TableServiceConfiguration.CONNECTION_STRING = connectionString;
    }

    @Value("${azure.tables.tableName}")
    public void setTableNameStatic(String tableName) {
        TableServiceConfiguration.TABLE_NAME = tableName;
    }

    @Bean
    public TableClient tableClientConfiguration() {
        return new TableClientBuilder()
                .connectionString(CONNECTION_STRING)
                .tableName(TABLE_NAME)
                .buildClient();
    }
    
}

You'll also need to add the following using statement at the top of the TableServiceConfig.java file.

import com.azure.data.tables.TableClient;
import com.azure.data.tables.TableClientBuilder;

6 - Implement Azure Cosmos DB table operations

All Azure Cosmos DB table operations for the sample app are implemented in the TablesServiceImpl class located in the Services directory. You'll need to import the com.azure.data.tables SDK package.

import com.azure.data.tables.TableClient;
import com.azure.data.tables.models.ListEntitiesOptions;
import com.azure.data.tables.models.TableEntity;
import com.azure.data.tables.models.TableTransactionAction;
import com.azure.data.tables.models.TableTransactionActionType;

At the start of the TableServiceImpl class, add a member variable for the TableClient object and a constructor to allow the TableClient object to be injected into the class.

@Autowired
private TableClient tableClient;

Get rows from a table

The TableClient class contains a method named listEntities which allows you to select rows from the table. In this example, since no parameters are being passed to the method, all rows will be selected from the table.

The method also takes a generic parameter of type TableEntity that specifies the model class data will be returned as. In this case, the built-in class TableEntity is used, meaning the listEntities method will return a PagedIterable<TableEntity> collection as its results.

public List<WeatherDataModel> retrieveAllEntities() {
    List<WeatherDataModel> modelList = tableClient.listEntities().stream()
        .map(WeatherDataUtils::mapTableEntityToWeatherDataModel)
        .collect(Collectors.toList());
    return Collections.unmodifiableList(WeatherDataUtils.filledValue(modelList));
}

The TableEntity class defined in the com.azure.data.tables.models package has properties for the partition key and row key values in the table. Together, these two values for a unique key for the row in the table. In this example application, the name of the weather station (city) is stored in the partition key and the date/time of the observation is stored in the row key. All other properties (temperature, humidity, wind speed) are stored in a dictionary in the TableEntity object.

It's common practice to map a TableEntity object to an object of your own definition. The sample application defines a class WeatherDataModel in the Models directory for this purpose. This class has properties for the station name and observation date that the partition key and row key will map to, providing more meaningful property names for these values. It then uses a dictionary to store all the other properties on the object. This is a common pattern when working with Table storage since a row can have any number of arbitrary properties and we want our model objects to be able to capture all of them. This class also contains methods to list the properties on the class.

public class WeatherDataModel {

    public WeatherDataModel(String stationName, String observationDate, OffsetDateTime timestamp, String etag) {
        this.stationName = stationName;
        this.observationDate = observationDate;
        this.timestamp = timestamp;
        this.etag = etag;
    }

    private String stationName;

    private String observationDate;

    private OffsetDateTime timestamp;

    private String etag;

    private Map<String, Object> propertyMap = new HashMap<String, Object>();

    public String getStationName() {
        return stationName;
    }

    public void setStationName(String stationName) {
        this.stationName = stationName;
    }

    public String getObservationDate() {
        return observationDate;
    }

    public void setObservationDate(String observationDate) {
        this.observationDate = observationDate;
    }

    public OffsetDateTime getTimestamp() {
        return timestamp;
    }

    public void setTimestamp(OffsetDateTime timestamp) {
        this.timestamp = timestamp;
    }

    public String getEtag() {
        return etag;
    }

    public void setEtag(String etag) {
        this.etag = etag;
    }

    public Map<String, Object> getPropertyMap() {
        return propertyMap;
    }

    public void setPropertyMap(Map<String, Object> propertyMap) {
        this.propertyMap = propertyMap;
    }
}

The mapTableEntityToWeatherDataModel method is used to map a TableEntity object to a WeatherDataModel object. The mapTableEntityToWeatherDataModel method directly maps the PartitionKey, RowKey, Timestamp, and Etag properties and then uses the properties.keySet to iterate over the other properties in the TableEntity object and map those to the WeatherDataModel object, minus the properties that have already been directly mapped.

Edit the code in the mapTableEntityToWeatherDataModel method to match the following code block.

public static WeatherDataModel mapTableEntityToWeatherDataModel(TableEntity entity) {
    WeatherDataModel observation = new WeatherDataModel(
        entity.getPartitionKey(), entity.getRowKey(),
        entity.getTimestamp(), entity.getETag());
    rearrangeEntityProperties(observation.getPropertyMap(), entity.getProperties());
    return observation;
}

private static void rearrangeEntityProperties(Map<String, Object> target, Map<String, Object> source) {
    Constants.DEFAULT_LIST_OF_KEYS.forEach(key -> {
        if (source.containsKey(key)) {
            target.put(key, source.get(key));
        }
    });
    source.keySet().forEach(key -> {
        if (Constants.DEFAULT_LIST_OF_KEYS.parallelStream().noneMatch(defaultKey -> defaultKey.equals(key))
        && Constants.EXCLUDE_TABLE_ENTITY_KEYS.parallelStream().noneMatch(defaultKey -> defaultKey.equals(key))) {
            target.put(key, source.get(key));
        }
    });
}

Filter rows returned from a table

To filter the rows returned from a table, you can pass an OData style filter string to the listEntities method. For example, if you wanted to get all of the weather readings for Shanghai between midnight July 1, 2021 and midnight July 2, 2021 (inclusive) you would pass in the following filter string.

PartitionKey eq 'Shanghai' and RowKey ge '2021-07-01 12:00 AM' and RowKey le '2021-07-02 12:00 AM'

You can view all OData filter operators on the OData website in the section Filter System Query Option

In the example application, the FilterResultsInputModel object is designed to capture any filter criteria provided by the user.

public class FilterResultsInputModel implements Serializable {

    private String partitionKey;

    private String rowKeyDateStart;

    private String rowKeyTimeStart;

    private String rowKeyDateEnd;

    private String rowKeyTimeEnd;

    private Double minTemperature;

    private Double maxTemperature;

    private Double minPrecipitation;

    private Double maxPrecipitation;

    public String getPartitionKey() {
        return partitionKey;
    }

    public void setPartitionKey(String partitionKey) {
        this.partitionKey = partitionKey;
    }

    public String getRowKeyDateStart() {
        return rowKeyDateStart;
    }

    public void setRowKeyDateStart(String rowKeyDateStart) {
        this.rowKeyDateStart = rowKeyDateStart;
    }

    public String getRowKeyTimeStart() {
        return rowKeyTimeStart;
    }

    public void setRowKeyTimeStart(String rowKeyTimeStart) {
        this.rowKeyTimeStart = rowKeyTimeStart;
    }

    public String getRowKeyDateEnd() {
        return rowKeyDateEnd;
    }

    public void setRowKeyDateEnd(String rowKeyDateEnd) {
        this.rowKeyDateEnd = rowKeyDateEnd;
    }

    public String getRowKeyTimeEnd() {
        return rowKeyTimeEnd;
    }

    public void setRowKeyTimeEnd(String rowKeyTimeEnd) {
        this.rowKeyTimeEnd = rowKeyTimeEnd;
    }

    public Double getMinTemperature() {
        return minTemperature;
    }

    public void setMinTemperature(Double minTemperature) {
        this.minTemperature = minTemperature;
    }

    public Double getMaxTemperature() {
        return maxTemperature;
    }

    public void setMaxTemperature(Double maxTemperature) {
        this.maxTemperature = maxTemperature;
    }

    public Double getMinPrecipitation() {
        return minPrecipitation;
    }

    public void setMinPrecipitation(Double minPrecipitation) {
        this.minPrecipitation = minPrecipitation;
    }

    public Double getMaxPrecipitation() {
        return maxPrecipitation;
    }

    public void setMaxPrecipitation(Double maxPrecipitation) {
        this.maxPrecipitation = maxPrecipitation;
    }
}

When this object is passed to the retrieveEntitiesByFilter method in the TableServiceImpl class, it creates a filter string for each non-null property value. It then creates a combined filter string by joining all of the values together with an "and" clause. This combined filter string is passed to the listEntities method on the TableClient object and only rows matching the filter string will be returned. You can use a similar method in your code to construct suitable filter strings as required by your application.

public List<WeatherDataModel> retrieveEntitiesByFilter(FilterResultsInputModel model) {

    List<String> filters = new ArrayList<>();

    if (!StringUtils.isEmptyOrWhitespace(model.getPartitionKey())) {
        filters.add(String.format("PartitionKey eq '%s'", model.getPartitionKey()));
    }
    if (!StringUtils.isEmptyOrWhitespace(model.getRowKeyDateStart())
            && !StringUtils.isEmptyOrWhitespace(model.getRowKeyTimeStart())) {
        filters.add(String.format("RowKey ge '%s %s'", model.getRowKeyDateStart(), model.getRowKeyTimeStart()));
    }
    if (!StringUtils.isEmptyOrWhitespace(model.getRowKeyDateEnd())
            && !StringUtils.isEmptyOrWhitespace(model.getRowKeyTimeEnd())) {
        filters.add(String.format("RowKey le '%s %s'", model.getRowKeyDateEnd(), model.getRowKeyTimeEnd()));
    }
    if (model.getMinTemperature() != null) {
        filters.add(String.format("Temperature ge %f", model.getMinTemperature()));
    }
    if (model.getMaxTemperature() != null) {
        filters.add(String.format("Temperature le %f", model.getMaxTemperature()));
    }
    if (model.getMinPrecipitation() != null) {
        filters.add(String.format("Precipitation ge %f", model.getMinPrecipitation()));
    }
    if (model.getMaxPrecipitation() != null) {
        filters.add(String.format("Precipitation le %f", model.getMaxPrecipitation()));
    }

    List<WeatherDataModel> modelList = tableClient.listEntities(new ListEntitiesOptions()
        .setFilter(String.join(" and ", filters)), null, null).stream()
        .map(WeatherDataUtils::mapTableEntityToWeatherDataModel)
        .collect(Collectors.toList());
    return Collections.unmodifiableList(WeatherDataUtils.filledValue(modelList));
}

Insert data using a TableEntity object

The simplest way to add data to a table is by using a TableEntity object. In this example, data is mapped from an input model object to a TableEntity object. The properties on the input object representing the weather station name and observation date/time are mapped to the PartitionKey and RowKey) properties respectively which together form a unique key for the row in the table. Then the additional properties on the input model object are mapped to dictionary properties on the TableClient object. Finally, the createEntity method on the TableClient object is used to insert data into the table.

Modify the insertEntity class in the example application to contain the following code.

public void insertEntity(WeatherInputModel model) {
    tableClient.createEntity(WeatherDataUtils.createTableEntity(model));
}

Upsert data using a TableEntity object

If you try to insert a row into a table with a partition key/row key combination that already exists in that table, you'll receive an error. For this reason, it's often preferable to use the upsertEntity instead of the insertEntity method when adding rows to a table. If the given partition key/row key combination already exists in the table, the upsertEntity method will update the existing row. Otherwise, the row will be added to the table.

public void upsertEntity(WeatherInputModel model) {
    tableClient.upsertEntity(WeatherDataUtils.createTableEntity(model));
}

Insert or upsert data with variable properties

One of the advantages of using the Azure Cosmos DB Tables API is that if an object being loaded to a table contains any new properties then those properties are automatically added to the table and the values stored in Azure Cosmos DB. There's no need to run DDL statements like ALTER TABLE to add columns as in a traditional database.

This model gives your application flexibility when dealing with data sources that may add or modify what data needs to be captured over time or when different inputs provide different data to your application. In the sample application, we can simulate a weather station that sends not just the base weather data but also some additional values. When an object with these new properties is stored in the table for the first time, the corresponding properties (columns) will be automatically added to the table.

In the sample application, the ExpandableWeatherObject class is built around an internal dictionary to support any set of properties on the object. This class represents a typical pattern for when an object needs to contain an arbitrary set of properties.

public class ExpandableWeatherObject {

    private String stationName;

    private String observationDate;

    private Map<String, Object> propertyMap = new HashMap<String, Object>();

    public String getStationName() {
        return stationName;
    }

    public void setStationName(String stationName) {
        this.stationName = stationName;
    }

    public String getObservationDate() {
        return observationDate;
    }

    public void setObservationDate(String observationDate) {
        this.observationDate = observationDate;
    }

    public Map<String, Object> getPropertyMap() {
        return propertyMap;
    }

    public void setPropertyMap(Map<String, Object> propertyMap) {
        this.propertyMap = propertyMap;
    }

    public boolean containsProperty(String key) {
        return this.propertyMap.containsKey(key);
    }

    public Object getPropertyValue(String key) {
        return containsProperty(key) ? this.propertyMap.get(key) : null;
    }

    public void putProperty(String key, Object value) {
        this.propertyMap.put(key, value);
    }

    public List<String> getPropertyKeys() {
        List<String> list = Collections.synchronizedList(new ArrayList<String>());
        Iterator<String> iterators = this.propertyMap.keySet().iterator();
        while (iterators.hasNext()) {
            list.add(iterators.next());
        }
        return Collections.unmodifiableList(list);
    }

    public Integer getPropertyCount() {
        return this.propertyMap.size();
    }
}

To insert or upsert such an object using the API for Table, map the properties of the expandable object into a TableEntity object and use the createEntity or upsertEntity methods on the TableClient object as appropriate.

public void insertExpandableEntity(ExpandableWeatherObject model) {
    tableClient.createEntity(WeatherDataUtils.createTableEntity(model));
}

public void upsertExpandableEntity(ExpandableWeatherObject model) {
    tableClient.upsertEntity(WeatherDataUtils.createTableEntity(model));
}

Update an entity

Entities can be updated by calling the updateEntity method on the TableClient object. Because an entity (row) stored using the Tables API could contain any arbitrary set of properties, it's often useful to create an update object based around a dictionary object similar to the ExpandableWeatherObject discussed earlier. In this case, the only difference is the addition of an etag property which is used for concurrency control during updates.

public class UpdateWeatherObject {

    private String stationName;

    private String observationDate;

    private String etag;

    private Map<String, Object> propertyMap = new HashMap<String, Object>();

    public String getStationName() {
        return stationName;
    }

    public void setStationName(String stationName) {
        this.stationName = stationName;
    }

    public String getObservationDate() {
        return observationDate;
    }

    public void setObservationDate(String observationDate) {
        this.observationDate = observationDate;
    }

    public String getEtag() {
        return etag;
    }

    public void setEtag(String etag) {
        this.etag = etag;
    }

    public Map<String, Object> getPropertyMap() {
        return propertyMap;
    }

    public void setPropertyMap(Map<String, Object> propertyMap) {
        this.propertyMap = propertyMap;
    }
}

In the sample app, this object is passed to the updateEntity method in the TableServiceImpl class. This method first loads the existing entity from the Tables API using the getEntity method on the TableClient. It then updates that entity object and uses the updateEntity method save the updates to the database. Note how the updateEntity method takes the current Etag of the object to ensure the object hasn't changed since it was initially loaded. If you want to update the entity regardless, you may pass a value of etag to the updateEntity method.

public void updateEntity(UpdateWeatherObject model) {
    TableEntity tableEntity = tableClient.getEntity(model.getStationName(), model.getObservationDate());
    Map<String, Object> propertiesMap = model.getPropertyMap();
    propertiesMap.keySet().forEach(key -> tableEntity.getProperties().put(key, propertiesMap.get(key)));
    tableClient.updateEntity(tableEntity);
}

Remove an entity

To remove an entity from a table, call the deleteEntity method on the TableClient object with the partition key and row key of the object.

public void deleteEntity(WeatherInputModel model) {
    tableClient.deleteEntity(model.getStationName(),
            WeatherDataUtils.formatRowKey(model.getObservationDate(), model.getObservationTime()));
}

7 - Run the code

Run the sample application to interact with the Azure Cosmos DB Tables API. The first time you run the application, there will be no data because the table is empty. Use any of the buttons at the top of application to add data to the table.

Selecting the Insert using Table Entity button opens a dialog allowing you to insert or upsert a new row using a TableEntity object.

Selecting the Insert using Expandable Data button brings up a dialog that enables you to insert an object with custom properties, demonstrating how the Azure Cosmos DB Tables API automatically adds properties (columns) to the table when needed. Use the Add Custom Field button to add one or more new properties and demonstrate this capability.

Use the Insert Sample Data button to load some sample data into your Azure Cosmos DB table.

Select the Filter Results item in the top menu to be taken to the Filter Results page. On this page, fill out the filter criteria to demonstrate how a filter clause can be built and passed to the Azure Cosmos DB Tables API.

Clean up resources

When you're finished with the sample application, you should remove all Azure resources related to this article from your Azure account. You can do this by deleting the resource group.

Next steps

In this quickstart, you've learned how to create an Azure Cosmos DB account, create a table using the Data Explorer, and run an app. Now you can query your data using the API for Table.