Python quickstart

View as Markdown

In about 10 minutes you install the M-Files gRPC client package for Python, configure a connection, and run a program that logs in to a vault and prints its 10 most recently modified documents.

Prerequisites

1. Install the package

The package mfiles-grpc contains the client code generated from the M-Files protos and the helpers that the guides use. In an empty folder, create a virtual environment, activate it and install the package:

python -m venv .venv
source .venv/bin/activate        # PowerShell: .venv\Scripts\Activate.ps1
pip install "mfiles-grpc[keyring]"

The later steps run in the same terminal, with the environment active.

It has two parts:

The keyring extra lets a browser sign-in be remembered; leave it out if you don't need that.

Versions. There is one package version per M-Files release: 26.11.0 has the API of M-Files 26.11, and 26.11.1 is a revision of it. A version works with its release and later ones; with an older server, the methods that the server does not have fail with UNIMPLEMENTED. To get the version of a release, pin it: pip install "mfiles-grpc==26.11.*". The package is a preview: its helpers can still change between releases.

2. Write the program

Save this as quickstart.py:

from mfiles_grpc import MFilesClient, run
from mfiles_grpc.values import is_not_deleted, is_object_type

from mfiles import IRPCSearch_pb2, IRPCSearch_pb2_grpc
from mfiles import enums_pb2 as enums
from mfiles import structs_pb2 as structs


def main() -> None:
    """Lists the 10 most recently modified documents of the vault."""
    # Connect and log in with the MFILES_* settings. Leaving the block logs out.
    with MFilesClient.from_env("M-Files gRPC quickstart") as client:
        # The search: what to find, in what order, and how many.
        request = IRPCSearch_pb2.SearchObjectsRequest(
            # Documents (object type 0) that are not deleted.
            conditions=[structs.SearchConditionArray(
                value=[is_object_type(0), is_not_deleted()])],
            # Newest first by property 21, "Last modified". The server sorts, then takes 10.
            sorting=structs.SearchResultsSorting(rules=structs.SortRuleValueArray(values=[
                structs.SortRule(key=21, options=enums.SortRuleFlags(sort_descending=True))])),
            limit=10)

        # Service stubs run on the client's channel, which adds the session to each call.
        search = client.stub(IRPCSearch_pb2_grpc.IRPCSearchStub)
        response = search.SearchObjects(request)

        # Print each document: its title, ID, version and when it was last modified.
        for item in response.results:
            version = item.object.version_info
            print(f"{version.title} (ID {item.object.object_info.obj_id.item_id.internal_id}, "
                  f"version {version.version.internal_version}, "
                  f"modified {version.last_modified_at_utc.ToDatetime():%Y-%m-%d %H:%M:%S} UTC)")


if __name__ == "__main__":
    run(main)

Full source: mfiles-grpc-python/examples/quickstart.py

3. Configure the connection

The helper reads environment variables. For M-Files Cloud with single sign-on (PowerShell):

$env:MFILES_ADDRESS = "myvault.cloudvault.m-files.com:443"
$env:MFILES_AUTH = "browser"

For an on-premises server without TLS, with a user name and password (bash):

export MFILES_ADDRESS=myserver:7766 MFILES_TLS=false
export MFILES_VAULT='{00000000-0000-0000-0000-000000000000}'
export MFILES_USERNAME=me
read -rs MFILES_PASSWORD && export MFILES_PASSWORD
Variable Meaning
MFILES_ADDRESS host:port of the gRPC endpoint
MFILES_TLS true (default) or false for a server without TLS
MFILES_CA_CERT Optional: PEM file of root certificates to trust, for a self-signed server certificate
MFILES_VAULT The vault GUID, with braces
MFILES_AUTH browser for single sign-on in the browser; then MFILES_VAULT is optional
MFILES_REMEMBER_SIGN_IN Optional, with MFILES_AUTH=browser: true keeps the sign-in in the operating system's credential store (the keyring extra)
MFILES_USERNAME, MFILES_PASSWORD An M-Files user account (M-Files authentication)
MFILES_TOKEN Instead of a password, for single sign-on: an access token from the vault's identity provider
MFILES_AUTH_PLUGIN, MFILES_AUTH_SCOPE Optional, with MFILES_TOKEN: the authentication plugin and its configuration scope. Without them the helper asks the vault and uses its plugin, if the vault has one or one marked as the default

Use MFILES_TLS=false only for a server that has no certificate, such as a lab installation.

Single sign-on. A vault that signs users in through M-Files Login Service, as is typical on M-Files Cloud, refuses a password login with "Authentication failed.". With MFILES_AUTH=browser the helper signs you in through the browser, as M-Files' own clients do, and logs in with the token (see M-Files Login Service). The vault comes from the sign-in configuration, so MFILES_VAULT is not needed. With MFILES_REMEMBER_SIGN_IN=true, the next run signs in without the browser. A token that you get elsewhere goes into MFILES_TOKEN.

4. Run it

python quickstart.py

The output lists the documents, newest first. In the M-Files Sample Vault it starts like this:

Project Plan (template) (ID 445, version 1, modified 2012-06-07 12:01:57 UTC)
Project Tasks - CRM Development (ID 444, version 1, modified 2012-06-07 11:54:29 UTC)
Corporate Presentation (ID 443, version 1, modified 2012-06-07 11:40:14 UTC)
...

The code

is_object_type and is_not_deleted are small builders from mfiles_grpc.values. They show the message shapes of the two conditions; lookups to object types use the value list ID -11:

def property_condition(property_def: int, condition_type: int,
                       value: structs.TypedValue) -> structs.SearchCondition:
    """A condition on a property value, e.g. property 100 (Class) equals a lookup.

    Args:
        property_def: the property definition's ID.
        condition_type: how to compare, a CONDITION_TYPE_* value.
        value: the value to compare with.
    """
    return structs.SearchCondition(
        expression=structs.Expression(
            type=enums.EXPRESSION_TYPE_PROPERTY_VALUE,
            data=structs.ExpressionUnion(property_value=structs.ExpressionUnion.PropertyValueData(
                property_def=property_def))),
        type=condition_type,
        value=value)


def status_condition(status: int, condition_type: int,
                     value: structs.TypedValue) -> structs.SearchCondition:
    """A condition on a status value, e.g. the object type or the deleted flag.

    Args:
        status: the status value, a STATUS_TYPE_* value.
        condition_type: how to compare, a CONDITION_TYPE_* value.
        value: the value to compare with.
    """
    return structs.SearchCondition(
        expression=structs.Expression(
            type=enums.EXPRESSION_TYPE_STATUS_VALUE,
            data=structs.ExpressionUnion(status_value=structs.ExpressionUnion.StatusValueData(
                type=status))),
        type=condition_type,
        value=value)


def is_object_type(object_type: int) -> structs.SearchCondition:
    """Objects of one object type. Value list -11 lists the object types.

    Args:
        object_type: the object type's ID, e.g. 0 for documents.
    """
    return status_condition(enums.STATUS_TYPE_OBJECT_TYPE, enums.CONDITION_TYPE_EQUAL,
                            lookup(-11, object_type))


def is_not_deleted() -> structs.SearchCondition:
    """Objects that are not deleted."""
    return status_condition(enums.STATUS_TYPE_IS_DELETED, enums.CONDITION_TYPE_EQUAL,
                            boolean(False))

Full source: mfiles-grpc-python/src/mfiles_grpc/values.py

More about conditions: Search.

What the helper does for you

MFilesClient (mfiles_grpc.client) hides the parts that the proto files cannot express.

It opens the channel, with TLS unless MFILES_TLS is false, and sets the message size limits that M-Files' own client uses: responses of any size, requests up to 48 MiB (the server's default limit):

# A TLS or insecure channel with M-Files' message size limits, through the interceptor.
options = [("grpc.max_receive_message_length", MAX_RECEIVE_MESSAGE_BYTES),
           ("grpc.max_send_message_length", MAX_SEND_MESSAGE_BYTES)]
if config.tls:
    root_certificates = Path(config.ca_cert).read_bytes() if config.ca_cert else None
    credentials = grpc.ssl_channel_credentials(root_certificates=root_certificates)
    self._channel = grpc.secure_channel(config.address, credentials, options=options)
else:
    self._channel = grpc.insecure_channel(config.address, options=options)
self.channel = grpc.intercept_channel(self._channel, _MetadataInterceptor(self))

Full source: mfiles-grpc-python/src/mfiles_grpc/client.py

It logs in with LogIn. The response's session_data.session_id (48 bytes) identifies the session:

# Log in to the vault, naming the client and how it authenticates.
request = IRPCLogin_pb2.LogInRequest(
    environment_data=structs_pb2.EnvironmentData(
        client_name=self.client_name, host_platform="Python"),
    client_data=self._client_data(),
    login_data=structs_pb2.LoginData(
        server_hostname=self.config.host, vault_guid=self.config.vault),
    authentication_data=authentication)
response = self.login.LogIn(request)
if not response.session_data.session_id:
    raise RuntimeError("LogIn did not complete: the authentication needs more steps")
# From now on, the interceptor sends this in "mfiles-session-id-bin" on every call.
self.session_id = response.session_data.session_id

Full source: mfiles-grpc-python/src/mfiles_grpc/client.py

With a user name and password, authentication_data is:

# M-Files credentials: an M-Files user's name and password.
authentication = structs_pb2.AuthDataClient(
    type=enums_pb2.AUTH_DATA_TYPE_CREDENTIALS,
    data=structs_pb2.AuthDataClientUnion(credentials=structs_pb2.Credentials(
        type=enums_pb2.CREDENTIALS_TYPE_MFILES, username=username, password=password)))

Full source: mfiles-grpc-python/src/mfiles_grpc/client.py

Its interceptor adds call metadata to every call: mfiles-is-remote-call: true, a new random 16-byte mfiles-activity-id-bin, and after login the session ID as mfiles-session-id-bin. None of the request messages has a field for the session. A call that needs a session and does not have one fails with UNAUTHENTICATED. M-Files Cloud sets the cookie mfilesmsm to keep a client on one server instance; gRPC does not keep cookies, so the interceptor stores set-cookie values from the response metadata and sends them back in a cookie header on every later call. See Call metadata.

When a call fails, the interceptor raises MFilesError, decoded from the mfiles-error-serialized-bin and mfiles-error-text-bin trailers: the gRPC status (status), the message (message), the M-Files error code of the top-level error (hresult; the last entry of the error stack that is not the generic 0x80040001), every code of the stack (codes) and has(code):

# The M-Files error from the trailers: the long message and the error stack.
self.long_message: str = trailers.get(ERROR_TEXT_KEY, b"").decode("utf-8", "replace")
self.stack = list(structs_pb2.RPCErrorInfoArray.FromString(
    trailers.get(ERROR_STACK_KEY, b"")).value)
self.codes = [entry.error_code & 0xFFFFFFFF for entry in self.stack]  # cause first
# The top-level error, skipping entries that only pass the error on.
self.hresult: Optional[int] = next(
    (code for code in reversed(self.codes) if code != E_PASS_THROUGH),
    self.codes[-1] if self.codes else None)

Full source: mfiles-grpc-python/src/mfiles_grpc/client.py

Catch it like this:

try:
    objects.GetObjectInfoAndProperties(request)
except MFilesError as error:
    if not error.has(0x8004000B):  # not found
        raise
    print(f"{error.hresult_hex}: {error.message}")

Errors lists the common codes and what to do about each.

At the end, close() calls LogOut and closes the channel.

If it does not work

Symptom Likely cause
UNAVAILABLE, "Tls handshake failed ... WRONG_VERSION_NUMBER" TLS against a server without TLS: set MFILES_TLS=false, only for a lab server on a trusted network
UNIMPLEMENTED, "Received http2 header with status: 404" A plaintext call to a TLS endpoint, such as M-Files Cloud: remove MFILES_TLS=false
UNIMPLEMENTED for one method, other calls work The server is older than the package and does not have the method
UNAVAILABLE, "Connection refused" Wrong host or port, or a firewall
UNKNOWN, M-Files code 0x8004001A, "Authentication failed." Wrong user name or password, or the vault uses single sign-on (see step 3)
UNKNOWN, M-Files code 0x80040020, "The specified document vault does not exist." Wrong MFILES_VAULT
UNAUTHENTICATED No session or an expired one: log in again

Next steps

Generate the stubs yourself

The package is the easiest way to the client code. To generate it yourself, for example with other grpcio-tools options, use the proto files of the gRPC API bundle. Include the files in mfiles/api/ too: the service files import mfiles/api/annotations.proto, and a mfiles/*.proto pattern misses it (the import then fails with No module named 'mfiles.api'):

python -m grpc_tools.protoc -I proto --python_out=gen --grpc_python_out=gen \
    proto/mfiles/*.proto proto/mfiles/api/*.proto

An M-Files installation also contains a proto file, in Common\Web\GRPC\proto under the installation folder. That is the legacy single combined file; it does not produce the module names that the package and the documentation use (mfiles.IRPCSearch_pb2, mfiles.structs_pb2). Prefer the bundle.