Authentication and sessions

View as Markdown

Almost every call needs a session. LogIn authenticates the caller and returns a session ID; no request message has a field for it. It travels in the binary call metadata mfiles-session-id-bin of every later call.

LogIn  ->  session_data.session_id (48 bytes)
every call  ->  metadata  mfiles-session-id-bin: <the 48 bytes, unchanged>

gRPC libraries base64-encode -bin metadata values themselves: pass the raw bytes. The value must be exactly the 48 bytes from session_data.session_id; other lengths are rejected. A session belongs to one vault, or to no vault (a server-level session).

Calls that need no session

QueryVersion, GetServerCapabilities, GetAuthenticationConfiguration, LogIn and a few public methods (for example those of IRPCSharedLinksPublic, which a shared link's access key authorizes) are served without a session. The reference marks each method with its session requirement.

Choosing the authentication method

To learn what a server or vault expects, call GetAuthenticationConfiguration anonymously. AUTH_METHOD_PLUGIN with plugin information means single sign-on through that plugin; AUTH_METHOD_UNSPECIFIED with no plugins means the server has no plugin configured for the target and you use credentials or Windows authentication.

Credentials and tokens travel in the request. Send them only over TLS (Connecting).

The login loop

Every type uses the same loop. Single-step types finish in the first call.

1. Call LogIn with environment_data, client_data, login_data and authentication_data.
2. If response.session_data.session_id is set, the login is complete.
3. Otherwise the server needs another step: send LogIn again with the next authentication data, and copy response.authentication_data.authentication_attempt_identifier into the authentication_attempt_identifier field of the union member you send (for example data.windows.authentication_attempt_identifier). Repeat from 2.

The server's half of the exchange is in response.authentication_data.data. For Windows and plugin authentication it carries the next token or values and is_done.

A failed authentication is reported as E_MFILES_AUTHENTICATION_FAILED ("Authentication failed.") without details: a wrong password, an unknown user and a login blocked by the server's login attack protection look the same. Repeated failures make the protection block further attempts for a while, so do not retry a failing login in a loop.

User name and password

One call. Fill authentication_data.data.credentials (username, password, type, and domain for Windows accounts) and set authentication_data.type to AUTH_DATA_TYPE_CREDENTIALS:

authentication = structs_pb2.AuthDataClient(
    type=enums_pb2.AUTH_DATA_TYPE_CREDENTIALS,
    data=structs_pb2.AuthDataClientUnion(credentials=structs_pb2.Credentials(
        type=enums_pb2.CREDENTIALS_TYPE_MFILES, username=username, password=password)))

Source: python/mfiles_client.py

request = IRPCLogin_pb2.LogInRequest(
    environment_data=structs_pb2.EnvironmentData(
        client_name=self.client_name, host_platform="Python"),
    client_data=self._client_data(),
    login_data=structs_pb2.LoginData(
        server_hostname=self.config.host, vault_guid=self.config.vault),
    authentication_data=authentication)
response = self.login.LogIn(request)
if not response.session_data.session_id:
    raise RuntimeError("LogIn did not complete: the authentication needs more steps")
# From now on, the interceptor sends this in "mfiles-session-id-bin" on every call.
self.session_id = response.session_data.session_id

Source: python/mfiles_client.py

Windows authentication (SSPI)

The client runs the Windows SSPI handshake and relays its tokens through LogIn. The server accepts tokens of the Negotiate package (Kerberos or NTLM); M-Files' C# client library uses NTLM. The loop:

1. Initialize an SSPI client context; it produces the first token.
2. LogIn with type: AUTH_DATA_TYPE_WINDOWS and data.windows.auth_data = the token. Leave authentication_attempt_identifier unset in the first call.
3. The response has no session yet. Feed response.authentication_data.data.windows.auth_data to SSPI to get the next token, and send it with data.windows.authentication_attempt_identifier = the attempt identifier from the response.
4. Stop when the response has session_data.session_id (and data.windows.is_done is true).

With NTLM and Negotiate against a local server, the login completed in two LogIn calls. If the Windows user has no account in the vault, the last call fails with E_MFILES_ACCESS_DENIED ("You do not have a user account in this document vault").

Single sign-on (plugin authentication)

The server does not run the identity provider's sign-in for you. Your client obtains a token from the identity provider and hands it to LogIn:

1. Discover, anonymously: GetAuthenticationConfiguration with host_name set to the host you connect to (or with vault_guid), and type set to ACCOUNT_TYPE_WINDOWS: users who sign in through an identity provider have Windows-type login accounts, and a vault can limit its configuration to that account type (configuration scope *:WINDOWS:). Such a vault returns no plugin when type is left out. The response names the plugin (plugin_infos[].name, for example MFLoginService.Default, assembly MFiles.AuthenticationProviders.OAuth). Its plugin_configuration holds the identity provider settings (AuthorizationEndpoint, TokenEndpoint, ClientID, Scope, RedirectURI, UsePkceWithAuthorizationCode, UseAccessTokenInWeb, ...) and its system_configuration holds Scope (the configuration scope) and VaultGUID.
2. Sign in at the identity provider with those settings (for OAuth: authorization code flow, with PKCE when UsePkceWithAuthorizationCode says so) and get the token.
3. Log in with type: AUTH_DATA_TYPE_PLUGIN and data.plugin:

plugin_data = structs_pb2.AuthDataClientUnion.PluginData(
    plugin_name=plugin,
    configuration_scope=scope,
    vault_guid=plugin_vault,
    authentication_attempt_identifier=structs_pb2.AuthenticationAttemptIdentifier(
        data=bytes(32)),
    data_format=enums_pb2.PLUGIN_AUTH_DATA_FORMAT_UNENCRYPTED,
    auth_data=[structs_pb2.NamedValue(key="Token", value=structs_pb2.TypedValue(
        type=enums_pb2.DATATYPE_TEXT, data=structs_pb2.TypedValueUnion(text=token)))])
authentication = structs_pb2.AuthDataClient(
    type=enums_pb2.AUTH_DATA_TYPE_PLUGIN,
    data=structs_pb2.AuthDataClientUnion(plugin=plugin_data))

Source: python/mfiles_client.py

A plugin can ask for more steps; the response then has data.plugin.is_done false and no session. Continue with the loop above. The session you get is the same kind as from a password login.

M-Files Login Service

M-Files Login Service is M-Files' own OpenID Connect sign-in service. It signs the user in with the identity provider of their organization (for example Microsoft Entra ID) and issues tokens for the vault. A vault that uses it returns the plugin MFLoginService.Default (assembly MFiles.AuthenticationProviders.OAuth) from GetAuthenticationConfiguration. When a vault offers it, use it: your application then signs users in the same way as M-Files' own clients.

The plugin's plugin_configuration has everything the sign-in needs. Read it from the vault every time; do not hard-code it, because the endpoints differ between environments and the client ID and scope belong to the vault:

system_configuration has Scope (for example *: or *:WINDOWS:) and VaultGUID, which go into LogIn.

def find_login_service(client: MFilesClient) -> tuple[structs_pb2.PluginInfo, dict, dict]:
    """The vault's Login Service plugin, its OAuth settings and its system configuration."""
    # Users who sign in through an identity provider have Windows-type login accounts. A vault can
    # limit its configuration to that account type, and then answers only when the type is given.
    response = client.login.GetAuthenticationConfiguration(
        IRPCLogin_pb2.GetAuthenticationConfigurationRequest(
            host_name=client.config.host, type=enums_pb2.ACCOUNT_TYPE_WINDOWS))
    for plugin in response.plugin_infos:
        if plugin.name.startswith(LOGIN_SERVICE_PREFIX):
            settings = {v.key: v.value.data.text for v in plugin.plugin_configuration.value}
            system = {v.key: v.value.data.text for v in plugin.system_configuration.value}
            return plugin, settings, system
    raise SystemExit("The vault does not offer M-Files Login Service; "
                     f"plugins: {[p.name for p in response.plugin_infos]}")

Source: python/recipes/login_service.py

Sign in. Run the OAuth 2.0 authorization code flow with PKCE in the user's browser. A desktop or command-line program receives the code on a loopback address: listen on http://127.0.0.1:<any free port> with the path of RedirectURI, and send that as redirect_uri. The Login Service accepts any port with the loopback IP address (as RFC 8252 describes for native apps), but not with the name localhost: an authorization request with http://localhost:<port>/... ends on the Login Service's error page.

def sign_in(settings: dict) -> dict:
    """Runs the authorization code flow with PKCE in the browser; returns the token response."""
    verifier = secrets.token_urlsafe(48)
    challenge = base64.urlsafe_b64encode(
        hashlib.sha256(verifier.encode()).digest()).rstrip(b"=").decode()
    state = secrets.token_urlsafe(16)
    received: dict = {}

    class Callback(http.server.BaseHTTPRequestHandler):
        def do_GET(self):
            received.update(urllib.parse.parse_qs(urllib.parse.urlsplit(self.path).query))
            self.send_response(200)
            self.send_header("Content-Type", "text/plain; charset=utf-8")
            self.end_headers()
            self.wfile.write(b"Signed in. You can close this window.")

        def log_message(self, *args):
            pass

    # A loopback IP address with any free port: the Login Service accepts it as the redirect
    # URI of a native client (RFC 8252). Keep the path of the configured RedirectURI.
    path = urllib.parse.urlsplit(settings["RedirectURI"]).path or "/"
    with http.server.HTTPServer(("127.0.0.1", 0), Callback) as server:
        redirect_uri = f"http://127.0.0.1:{server.server_port}{path}"
        webbrowser.open(settings["AuthorizationEndpoint"] + "?" + urllib.parse.urlencode({
            "client_id": settings["ClientID"],
            "response_type": "code",
            "scope": settings["Scope"],
            "redirect_uri": redirect_uri,
            "state": state,
            "code_challenge": challenge,
            "code_challenge_method": "S256",
        }))
        print("Sign in in the browser window that opened...")
        server.timeout = 300
        server.handle_request()  # Waits for the browser to come back with the code.

    if received.get("state") != [state] or "code" not in received:
        raise SystemExit(f"Sign-in failed: {received.get('error', ['no code'])[0]}")

    # Exchange the code for tokens. A native client has no secret: PKCE proves that it started
    # the flow.
    body = urllib.parse.urlencode({
        "grant_type": "authorization_code",
        "code": received["code"][0],
        "redirect_uri": redirect_uri,
        "client_id": settings["ClientID"],
        "code_verifier": verifier,
    }).encode()
    with urllib.request.urlopen(urllib.request.Request(settings["TokenEndpoint"], body)) as reply:
        return json.load(reply)

Source: python/recipes/login_service.py

Log in with the access token, as described for plugin authentication above, with the plugin's name, its configuration scope and vault:

# The plugin validates the access token unless the configuration says to use the ID
# token instead.
use_id_token = settings.get("UseIdTokenAsAccessToken", "false").lower() == "true"
token_name = "id_token" if use_id_token else "access_token"
client.config.vault = client.config.vault or system["VaultGUID"]
session = client.log_in_with_token(tokens[token_name], plugin.name, system["Scope"],
                                   system.get("VaultGUID", ""))

Source: python/recipes/login_service.py

Log in again later without the browser: when the session has expired, get new tokens with the refresh token (grant_type=refresh_token, the refresh token and client_id; no secret and no PKCE) and call LogIn with the new access token.

def refresh(settings: dict, refresh_token: str) -> dict:
    """New tokens for a refresh token (the scope "offline_access" asks for one).

    The reply has a new refresh token, and the one passed in stops working: store the new one
    before using the new access token, and let only one process refresh at a time.
    """
    body = urllib.parse.urlencode({
        "grant_type": "refresh_token",
        "refresh_token": refresh_token,
        "client_id": settings["ClientID"],
    }).encode()
    with urllib.request.urlopen(urllib.request.Request(settings["TokenEndpoint"], body)) as reply:
        return json.load(reply)

Source: python/recipes/login_service.py

The whole program is recipes/login_service.py in the Python samples.

Programs that run without a person

A service or scheduled job cannot open a browser. With M-Files Login Service it signs in once interactively and then renews its tokens with the refresh token:

• Read the access token's lifetime from expires_in in each token response; do not assume a fixed value. LogIn accepts a refreshed access token like the first one.
• A refresh can return a new refresh token that replaces the one used. A replaced refresh token fails with HTTP 400 invalid_grant; the newest one keeps working.

So a program that renews logins unattended must:

• Store the new refresh token first, durably and atomically, before it uses the new access token. If the new token is lost, only an interactive sign-in can start the chain again. A refresh token signs in as the user without a browser, so keep it out of files and logs: a desktop tool can use the operating system's credential store (Windows Credential Manager, the macOS keychain; in Python, the keyring package). Store only the refresh token there and get a new access token at start-up; Windows Credential Manager holds at most 2,560 bytes per entry.
• Let one process refresh. Two processes that refresh with the same token race: one of them gets invalid_grant. Use a lock, or one process that hands out access tokens.
• Renew before the access token expires, and log in again with the fresh token when the session ends (UNAUTHENTICATED).

The refresh token's own lifetime, and how conditional-access policies of the user's organization affect it, are set by the Login Service and the identity provider; plan for an occasional interactive sign-in.

Anonymous

type: AUTH_DATA_TYPE_ANONYMOUS with an empty data.anonymous. It succeeds only for vaults that allow anonymous access; otherwise it fails with E_MFILES_AUTHENTICATION_FAILED.

Binding the session to a vault

login_data.vault_guid decides what the session is for:

Set login_data.server_hostname to the host name you connected to. On servers with vault-level login accounts (M-Files Cloud), a server-level login uses it to find the vault whose accounts apply.

Two ways to reach a vault:

• LogIn with the vault GUID. One authentication, one session, one round trip. This is the simplest path for a program that works with one known vault, and what M-Files' C# client library does.
• Server-level LogIn, then LogInWithSession. LogInWithSession takes an existing session in mfiles-session-id-bin and a login_data with the vault GUID, and returns a new session for that vault without authenticating again. The original session stays valid; log out both. Use this when you first need to list the vaults, or want sessions to several vaults from one sign-in (for example one SSO token). M-Files' web client logs in this way.

Both were verified against a server: the vault session from LogInWithSession has its own 48-byte ID, and logging it out leaves the server-level session alive.

Keeping the session alive

session_data.keep_alive_interval_in_seconds (450 seconds on a server with default settings) is how often to call KeepAlive. The server drops a session after its session timeout without a KeepAlive call: 15 minutes by default (twice the interval). A vault session can ask for a longer timeout, at most 7 days, with login_data.client_session_timeout_in_seconds; a shorter or longer value is ignored. Other calls do not extend the session: a program that is busy making calls must still call KeepAlive on a timer.

Upload sessions have their own interval, keep_upload_session_alive_interval_in_seconds; see Files.

When a call answers UNAUTHENTICATED

UNAUTHENTICATED means the server does not accept the session: it was never valid, expired, was logged out, the mfiles-session-id-bin header is missing, or it appears more than once. The status carries a fixed message ("The current session with M-Files Server is no longer valid...") and no error trailers.

Log in again with LogIn and continue with the new session_id.

Logging out

LogOut with the session in mfiles-session-id-bin ends the session and releases the license it holds; a session that is not logged out holds its license until it times out. Every later call with that session ID, including KeepAlive and a second LogOut, answers UNAUTHENTICATED.

Session assertions

Some methods also accept a session assertion in the metadata x-mf-session-assertion instead of mfiles-session-id-bin (SESSION_REQUIREMENT_SESSION_OR_ASSERTION in the reference). Session assertions are for M-Files' own services; applications use the session ID.