Store

Purpose of Store

The Store serves as a mediator for your application’s data flow. It provides efficient and consistent data management. Its primary purposes are:

• Data Orchestration: Manages the flow of data between the network, memory cache, and local storage (SourceOfTruth).
• Efficient Caching: Handles in-memory and disk caching strategies to optimize data retrieval, reduce latency, and minimize unnecessary network requests.
• Data Consistency: Guarantees consistency across all local data sources by synchronizing updates and, if a Validator is provided, ensuring that stale or invalid data is not served to consumers.
• Flexible Validation: Provides configurable validation mechanisms to ensure data integrity and freshness according to your app’s specific needs.

APIs

Store

Store has the following structure:

interface Store<Key : Any, Output : Any> {
  fun stream(request: StoreReadRequest<Key>): Flow<StoreReadResponse<Output>>
  suspend fun clear(key: Key)
  suspend fun clearAll()
}

Parameter

Key

Type

Any

Required

Required

Description

The type representing the key used to identify the data item.

Parameter

Output

Type

Any

Required

Required

Description

The type representing the domain data model representation of the item being retrieved.

stream

A function that returns a Flow of StoreReadResponse.

Parameter

request

Type

StoreReadRequest<Key>

Required

Required

Description

The request configuration for the data retrieval.

clear

A function that clears the data item identified by the given key.

Parameter

key

Type

Key

Required

Required

Description

The key identifying the data item to be cleared.

clearAll

A function that clears all data items.

Key Components

The RealStore is the default implementation of the Store interface. It’s composed of the following components:

1. FetcherController: Responsible for efficient network operations.

   • Prevents duplicate network calls for the same data.
   • Shares responses among multiple requesters.
   • Manages network request lifecycles.

2. SourceOfTruthWithBarrier: Wraps the SourceOfTruth.

   • Synchronizes read and write operations.
   • Provides persistent data storage.
   • Maintains data consistency.

3. Memory Cache: Fast, temporary storage.

   • Provides quick data retrieval without hitting disk or network.
   • Reduces latency.
   • Automatically manages memory usage.

4. Converter: Transforms data between network, local database, and domain data model types.

   • Facilitates data compatibility between different layers of the Store.

5. Validator: Validates cached data to ensure it’s still valid.

   • Prevents serving stale or invalid data to consumers.

Data Flow

Here’s how Store manages data flow through your app:

Reading Data

1. 1.Memory Cache Check
   1

   Memory Cache Check

   Checks if the requested data is present and valid in the in-memory cache.

   kotlinCopy

   val cachedToEmit = if (request.shouldSkipCache(CacheType.MEMORY)) {
       null
   } else {
       val output: Output? = memCache?.getIfPresent(request.key)
       val isInvalid = output != null && validator?.isValid(output) == false
       when {
           output == null || isInvalid -> null
           else -> output
       }
   }

   kotlinCopy

   cachedToEmit?.let { it: Output ->
     emit(StoreReadResponse.Data(value = it, origin = StoreReadResponseOrigin.Cache))
   }

   1. A.Validation
      A

      Validation

      If data is found, it's validating using the Validator. If a Validator is not provided, the data is considered valid.

   2. B.Emission
      B

      Emission

      Valid data is emitted immediately to the consumer.

2. 2.Decide Data Source
   2

   Decide Data Source

   Determines whether to read from the SourceOfTruth, fetch from the network, or both. Emits NoNewData if neither SourceOfTruth nor network fetch is requested.

   kotlinCopy

   if (sourceOfTruth == null && !request.fetch) {
     emit(StoreReadResponse.NoNewData(origin = StoreReadResponseOrigin.Cache))
     return@flow
   }

3. 3.Source of Truth Read
   3

   Source of Truth Read

   If a SourceOfTruth is configured and not skipped, attempts to read data from it.

   kotlinCopy

   if (request.fetch) {
     diskNetworkCombined(request, sourceOfTruth)
   } else {
     sourceOfTruth.reader(request.key, diskLock).transform { response -> ... }
   }

   1. A.Validation
      A

      Validation

      If data is found, it's validating using the Validator. If a Validator is not provided, the data is considered valid.

   2. B.Emission
      B

      Emission

      Valid data is emitted to the consumer.

   3. C.Network Fetch Decision
      C

      Network Fetch Decision

      If data is not found or invalid, decides whether to fetch from the network.

4. 4.Network Fetch
   4

   Network Fetch

   If a network fetch is required, fetches data from the network.

   kotlinCopy

   val networkFlow = createNetworkFlow(request, networkLock)

   1. A.Data Handling
      A

      Data Handling

      Fetched data is written to the SourceOfTruth and memory cache.

5. 5.Combine and Emit Data
   5

   Combine and Emit Data

   Combines the flows from the network and the source of truth to emit data in the correct order and origin to the subscribers.

   kotlinCopy

   emitAll(
     stream.transform { output: StoreReadResponse<Output> ->
       emit(output)
       // Update memory cache if needed
     },
   )

Writing Data

While the Store primarily focuses on reading data, it provides an internal write method for updating data in the cache and SourceOfTruth.

1. 1.Update Memory Cache
   1

   Update Memory Cache

   Updates the in-memory cache with the new data.

   kotlinCopy

   memCache?.put(key, value)

2. 2.Update Source of Truth
   2

   Update Source of Truth

   Writes the new data to the local storage (source of truth).

   kotlinCopy

   sourceOfTruth?.write(key, converter.fromOutputToLocal(value))

3. 3.Error Handling
   3

   Error Handling

   Catches any exceptions during the write operation and returns the appropriate result.

   kotlinCopy

   catch (error: Throwable) {
     StoreDelegateWriteResult.Error.Exception(error)
   }

Best Practices

• Configure Memory Usage: Set appropriate memory cache sizes based on device capabilities and data volume. Implement cache eviction policies that align with your app’s data freshness requirements.
• Implement Error Handling: Define clear error recovery paths for network failures, cache misses, and data corruption. Use the Fetcher retry mechanisms for transient network failures.
• Ensure Data Consistency: Set up the Source Of Truth as the single source of truth for critical data. Implement validation rules that catch data inconsistencies early.
• Optimize Network Usage: Batch related requests where possible. Configure appropriate cache TTLs to minimize unnecessary network calls.
• Monitor Performance: Track cache hit rates, network request frequencies, and data refresh patterns. Adjust caching strategies based on real-world usage patterns.
• Structure Keys Effectively: Design cache keys that are both unique and logical, avoiding collisions while maintaining readability. Consider namespacing keys for different data types.