Pull Streaming Channels

Cisco DNA Spaces supports the following pull channels:

HTTP

Your application can use HTTP Pull channel to retrieve events over HTTP/2 or over HTTP 1.x protocol. Your application initiates a HTTP GET to Cisco DNA Spaces Firehose API HTTP end-point. Events are continuously sent as they happen as a response to the GET request as long as the HTTP connection is active.

  • Your application needs to support secure (HTTPS) connections to the endpoint.

  • Events are encoded as JSON. Events are separated by a newline character. For sample JSON format events, see Sample Events JSON format.

  • Your application must authenticate using an API key.

    • API key is passed to the HTTP endpoint as X-API-KEY header.

    • API keys are provided in the Cisco DNA Spaces Partner Dashboard.

    • Cisco DNA Spaces Firehose API provides Production/Sandbox/Staging API keys.

    • The sandbox or staging key must be used during development or testing to ensure events to the production application are not diverted to your development or test instance of your application.

  • In case of an on-premise application, if you need the copy of the stream (replica) to be handled by the standby or secondary instances, you can use the replicaId (default value 1) query parameter to get the copy of the Firehose stream, where replicaId is expected as an integer value.

    Note
    replicaId is supported only for on-premise applications

  • Your application can request to replay events from a specific timestamp. This is done using the fromTimestamp parameter.

    • Timestamp is provided as the number of milliseconds since epoch.

    • If the parameter is not specified, HTTP Pull endpoint will only send events that are received after the HTTP connection is established.

    • All events have a unique identifier (record UID) that can be used to de-duplicate events.

    • In production deployments, it is recommended that your application uses this value in EventsStreamRequest (in conjunction with a dedupe) during a restart to avoid missing events.

    • Cisco DNA Spaces Firehose API maintains a rolling window of recent events for the partner. The width of the rolling window is currently unspecified and is subject to change. However, providing timestamps within the preceding hour are expected to work.

  • To distribute load across multiple receivers within your application or to increase throughput for high latency connections, your application can choose to make multiple connections to the Firehose API HTTP endpoint.

    • Your application can have up to 12 receivers.

    • When using more than one connection, your application is required to specify the partitions to receive on each connection using minPartition (default value 1) and maxPartition (default value 12) parameters to the GET request.

For example, if you have 4 receivers within your application, you would have the following:

  • Receiver #1 with minPartition = 1 and maxPartition = 3

  • Receiver #2 with minPartition = 4 and maxPartition = 6

  • Receiver #3 with minPartition = 7 and maxPartition = 9

  • Receiver #4 with minPartition = 10 and maxPartition = 12

    • Explicit partitioning as described above allows your application to perform a rolling upgrade. Since partition assignments remain stable, load is not moved to other receivers during the upgrade. Each receiver can use the replay parameter described in sectionfromTimestamp above to replay events for the assigned partitions without impacting events being sent to other receivers.

  • Keep Alive event is sent for every 15 seconds when no other event is sent.

  • Cisco DNA Spaces Firehose API HTTP Channel endpoint is at https://partners.dnaspaces.io/api/partners/v1/firehose/events and https://partners.dnaspaces.eu/api/partners/v1/firehose/events (for EU region). Use the curl as shown below and these URLs cannot be opened in browser.

  • To get a quick preview of the incoming events, use the below curl command after replacing <API_KEY> with your respective one.

    curl "https://partners.dnaspaces.io/api/partners/v1/firehose/events" -H "X-API-Key: <API_KEY>"

    curl "https://partners.dnaspaces.eu/api/partners/v1/firehose/events" -H "X-API-Key: <API_KEY>" (for EU region)

gRPC

gRPC is a popular high-performance, open source universal RPC framework. gRPC supports code generation in several languages and allows developers to start using the API without having to worry about implementation details. Your application requests events from the gRPC channel using a “Server streaming RPC call”. Your application sends a request to Firehose API gRPC channel, and receives a sequence of events back

  • Cisco DNA Spaces Firehose API gRPC service definition is provided as a .proto file. You can compile this file using the protocol buffer compiler (language specific).

  • gRPC operates over HTTP/2.

  • Events are encoded as Protocol Buffers (protobuf).

    Note

    For detailed Protobuf spec, see gRPC Service Spec/Protobuf Definition.

  • Your application must authenticate using an API key.

  • The API key is passed as a part of gRPC metadata with X-API-KEY as metadata name and the API key as the value.

    • API keys are provided in the Cisco DNA Spaces Partner Dashboard.

    • Cisco DNA Spaces Firehose API provides Production/Sandbox/Staging API keys.

    • Sandbox/Staging key must be used during development/testing to ensure events to production application is not diverted to your development/test instance of your application.

  • Your application can request to replay events from a specific timestamp. This is done using fromTimestamp value in the EventsStreamRequest.

    • Timestamp is provided as the number of milliseconds since epoch.

    • If the parameter is not specified, gRPC endpoint will only send events that are received after the gRPC request is received.

    • All events have a unique identifier (record UID) that can be used to de-duplicate events.

    • In production deployments, it is recommended that your application uses this value in the EventsStreamRequest (in conjunction with a dedupe) during a restart to avoid missing events.

    • The Cisco DNA Spaces Firehose API maintains a rolling window of recent events for the partner. The width of the rolling window is currently unspecified and is subject to change. However, providing timestamps within the preceding hour are expected to work.

  • If your application uses multiple receivers, you need to use the min_partition and max_partition values in the EventsStreamRequest.

  • Your application can have up to 12 receivers.

For example, if you have 4 receivers within your application, you would have the following:

  • Your application can request to replay events from a specific timestamp. This is done using the fromTimestamp parameter.

  • Receiver #1 with min_partition = 1 and max_partition = 3

  • Receiver #2 with min_partition = 4 and max_partition = 6

  • Receiver #3 with min_partition = 7 and max_partition = 9

  • Receiver #4 with min_partition = 10 and max_partition = 12

    • Explicit partitioning as described above allows your application to perform a rolling upgrade. Since partition assignments remain stable, load is not moved to other receivers during the upgrade. Each receiver can use the replay parameter described in section fromTimestamp above to replay events for the assigned partitions without impacting events being sent to other receivers.

  • The Keep Alive event is sent every 15 seconds, when no other event is sent.

  • The Cisco DNA Spaces gRPC Channel is available at the following endpoint:

    • Domain: partners.dnaspaces.io / partners.dnaspaces.eu(for EU region)

    • Port: 443