Skip to main content

HTTPS gateways and incoming requests

Advanced
Tutorial

Overview​

Canisters running on ICP can use HTTP requests in two ways: incoming and outgoing. Incoming HTTP requests refer to HTTP requests that are sent to a canister and can be used to retrieve data from a canister or send new data to the canister. Outgoing HTTP requests refer to HTTP requests that the canister sends to other canisters or external services to retrieve data or send new data.

HTTP gateways​

The ICP HTTP gateway protocol enables conventional HTTP clients to interact with the ICP network. HTTP gateways run adjacent to ICP and provide a connection used by software such as web browsers to send and receive standard HTTP requests and responses, including static assets, such as HTML, JavaScript, images, or videos. The HTTP gateway makes this workflow possible by translating standard HTTP requests into API canister calls that ICP canisters can understand and vice versa for HTTP responses.

There are several HTTP gateway instances maintained for ICP, but it should be noted that an HTTP gateway is a centralized component that could become compromised, creating a threat for users receiving HTTP content.

DFINITY exclusively controls the HTTP gateways that serve canisters on ic0.app and icp0.io. Developers can then also configure their own custom domains to point at the DFINITY controlled HTTP gateways.

Alternatively, developers can run their own HTTP gateways on their custom domains instead of using the DFINITY controlled gateways, but this is not well supported yet.

For a more secure solution, it is possible to run your own HTTP gateway instance locally.

Terminology​

  • Request: A message sent to a server or endpoint.

  • Client: A service that is able to send an HTTP formatted request and receive a response from a server.

  • Gateway: Enables the transfer of inbound and outbound messages.

HTTP request lifecycle​

On ICP, an HTTP request goes through the following lifecycle:

  1. An HTTP client makes an outbound request.

  2. The HTTP gateway intercepts the request and resolves the canister ID of the request's destination.

  3. The request is encoded with Candid and sent in a query call to the destination canister's http_request method.

  4. This request is sent to an ICP API boundary node, which will then forward the request to a replica node on the relevant subnet.

  5. The canister receives and processes the request, then returns the response.

  6. The HTTP gateway decodes the Candid encoding on the response.

  7. If the canister requests it, the gateway sends the request again via an update call to the canister's http_request_update method. The update call goes through consensus on the subnet.

  8. If the response size exceeds the maximum response size, the HTTP gateway fetches additional response body data through query calls.

  9. The HTTP gateway validates the response's certificate, if applicable.

  10. The HTTP gateway returns the decoded response to the HTTP client.

Additional details can be found in the HTTP gateway specification.

Running a local HTTP gateway​

To run your own local HTTP gateway, a proof-of-concept implementation can be used that enables a secure end-to-end connection with canisters deployed on ICP.

This implementation features:

  • Translation between HTTP asset requests and ICP API calls.

  • Detects ICP domains from principals and custom domain records.

  • Terminates TLS connections locally.

  • Bypasses remote gateway denylists.

  • Resolves crypto domains.

Installation​

You can download the pre-built installation package for your operating system.

Once installed, you will have the option to start or stop the IC HTTP proxy service. Start the service to begin using it. Once the proxy is running, it will handle all traffic on your computer. Any traffic that's not meant for the ICP mainnet will pass through the gateway, and traffic that is meant for ICP will be intercepted by the proxy. The proxy will log traffic in the file $HOME/Library/Preferences/dfinity/ichttpproxy/ic-http-proxy-proxy.log on macOS machines, or in the tmp directory on Windows machines.

For example, make the following curl request to the NNS:

curl -v https://nns.ic0.app

This will result in a log entry of:

{"level":30, "time": "2024-06-17T13:46:40.947Z","pid":43956,"hostname":"JessieMgeonsMBP.attlocal.net","name":"IC HTTP Proxy Server","msg":"Proxying web3 request for nns.ic0.app:443"}

Outgoing HTTP requests​

For outgoing HTTP requests, the HTTPS outcalls feature should be used.

Incoming HTTP requests​

To handle incoming HTTP requests, canisters must define methods for http_requests and http_requests_update for GET and POST requests respectively.

All HTTP requests are handled by the ICP HTTP Gateway, therefore you cannot make direct POST calls to a canister's http_request_update method with HTTP clients such as curl. Instead, you can make a POST call to a canister's HTTP endpoint, then configure the canister's http_request method to upgrade the call to http_request_update if necessary. Below is an example POST call to a canister's endpoint:

curl -X POST -H "Content-Type: application/json" -d '{"key":"value"}' https://<canister-id>.raw.ic0.app/<endpoint>

GET requests​

HTTP GET requests are used to retrieve and return existing data from an endpoint. To handle a canister's incoming GET requests, the http_request method can be exposed. Users and other services can call this method using a query call. To return HTTP response data, the following examples display how to configure the http_request to return an HTTP GET request.

In Motoko, a case configuration can be used to return different GET responses based on the endpoint.

Check out the certified cache example project to see an example of this implementation.

POST requests​

HTTP POST requests are used to send data to an endpoint with the intention of retaining that data. To handle incoming POST requests, the http_request_update method can be used. This method uses an update call, which can be used to change a canister's state. The following examples display how to configure http_request_update method within your canister.

In Motoko, a case configuration can be used to return different POST responses based on the endpoint.

Check out the certified cache example project to see an example of this implementation.

Serving HTTP requests​

Canisters can serve or handle an incoming HTTP request using the HTTP Gateway Protocol.

This allows developers to host web applications and APIs from a canister.

The following example returns 'Hello, World!' in the body at the /hello endpoint.

import HashMap = "mo:base/HashMap";
import Blob = "mo:base/Blob";
import Text "mo:base/Text";
import Option "mo:base/Option";

actor {

    public type HttpRequest = {
        body: Blob;
        headers: [HeaderField];
        method: Text;
        url: Text;
    };

    public type ChunkId = Nat;
    public type SetAssetContentArguments = {
        chunk_ids: [ChunkId];
        content_encoding: Text;
        key: Key;
        sha256: ?Blob;
    };
    public type Path = Text;
    public type Key = Text;

    public type HttpResponse = {
        body: Blob;
        headers: [HeaderField];
        status_code: Nat16;
    };

    public type HeaderField = (Text, Text);

    private func removeQuery(str: Text): Text {
        return Option.unwrap(Text.split(str, #char '?').next());
    };

    public query func http_request(req: HttpRequest): async (HttpResponse) {
        let path = removeQuery(req.url);
        if(path == "/hello") {
            return {
                body = Text.encodeUtf8("root page :" # path);
                headers = [];
                status_code = 200;
            };
        };

        return {
            body = Text.encodeUtf8("404 Not found :" # path);
            headers = [];
            status_code = 404;
        };
    };
}

Resources​