Python quickstart

View as Markdown

In about 15 minutes you generate the Python stubs of the M-Files gRPC API, configure a connection, and run a program that logs in to a vault and prints its 10 most recently modified documents.

Prerequisites

• Python 3.10 or newer.
• A vault you can reach and its GUID, with braces ({7BBEA0BC-40D8-4BCB-A67D-8A3ACFBDB575}). Don't know the GUID? Find the vault by its name lists the vaults you can log in to, or ask your M-Files administrator. The endpoint:
  • M-Files Cloud: the vault's host name; the gRPC endpoint is that host on port 443, with TLS.
  • On-premises: the M-Files Server host and its gRPC port, 7766 unless your administrator changed it. See Cloud and on-premises.
• A way to log in: an M-Files user account with a password, or single sign-on through M-Files Login Service. See Authentication and sessions.

1. Get the proto files

Download the gRPC API bundle and unpack it. Its proto folder is the published API definition: one file per service (mfiles/IRPCSearch.proto, ...) plus the shared mfiles/structs.proto and mfiles/enums.proto, all in the package MFiles, with documentation comments.

An M-Files installation also contains a proto file, in Common\Web\GRPC\proto under the installation folder. That is the legacy single combined file. Prefer the bundle: these samples and all code in the documentation use the bundle's file and module names (mfiles.IRPCSearch_pb2, mfiles.structs_pb2), which the combined file does not produce.

2. Generate the stubs

The samples are in the samples/python folder of the bundle, next to the proto folder.

cd samples/python
python -m venv .venv
.venv/bin/pip install -r requirements.txt     # Windows: .venv\Scripts\pip install ...
.venv/bin/python generate.py                  # Windows: .venv\Scripts\python generate.py

generate.py runs grpc_tools.protoc on every file of the bundle and writes the stubs into gen/. It finds the bundle at ../../proto; pass another folder as an argument or set MFILES_PROTO_DIR. The pinned versions are in requirements.txt (grpcio, grpcio-tools, protobuf).

To generate the stubs without generate.py, include the files in mfiles/api/ too: the service files import mfiles/api/annotations.proto, and a mfiles/*.proto pattern misses it (the import then fails with No module named 'mfiles.api'):

python -m grpc_tools.protoc -I proto --python_out=gen --grpc_python_out=gen \
    proto/mfiles/*.proto proto/mfiles/api/*.proto

3. Configure the connection

The samples read environment variables. For M-Files Cloud (PowerShell):

$env:MFILES_ADDRESS = "myvault.example.com:443"
$env:MFILES_VAULT = "{00000000-0000-0000-0000-000000000000}"
$env:MFILES_USERNAME = "me"
$env:MFILES_PASSWORD = Read-Host -Prompt 'Password'

For an on-premises server without TLS (bash):

export MFILES_ADDRESS=myserver:7766 MFILES_TLS=false
export MFILES_VAULT='{00000000-0000-0000-0000-000000000000}'
export MFILES_USERNAME=me
read -rs MFILES_PASSWORD && export MFILES_PASSWORD

Use MFILES_TLS=false only for a server that has no certificate, such as a lab installation.

Single sign-on. A vault that signs users in through M-Files Login Service, as is typical on M-Files Cloud, refuses a password login with "Authentication failed.". Set MFILES_TOKEN to an access token from the Login Service, and MFILES_AUTH_PLUGIN and MFILES_AUTH_SCOPE to the plugin and scope that the vault returns. M-Files Login Service shows how to get the token; the Python sample recipes/login_service.py does it end to end.

4. Run it

.venv/bin/python quickstart.py

The output lists the documents, newest first. In the M-Files Sample Vault it starts like this:

Project Plan (template) (ID 445, version 1, modified 2012-06-07 12:01:57 UTC)
Project Tasks - CRM Development (ID 444, version 1, modified 2012-06-07 11:54:29 UTC)
Corporate Presentation (ID 443, version 1, modified 2012-06-07 11:40:14 UTC)
...

The code

from mfiles_client import MFilesClient, run  # Also makes the stubs in gen/ importable.
from mfiles_values import is_not_deleted, is_object_type

from mfiles import IRPCSearch_pb2, IRPCSearch_pb2_grpc
from mfiles import enums_pb2 as enums
from mfiles import structs_pb2 as structs


def main() -> None:
    # Connect and log in with the MFILES_* settings. Leaving the block logs out.
    with MFilesClient.from_env("M-Files gRPC quickstart") as client:
        request = IRPCSearch_pb2.SearchObjectsRequest(
            # Documents (object type 0) that are not deleted.
            conditions=[structs.SearchConditionArray(
                value=[is_object_type(0), is_not_deleted()])],
            # Newest first by property 21, "Last modified". The server sorts, then takes 10.
            sorting=structs.SearchResultsSorting(rules=structs.SortRuleValueArray(values=[
                structs.SortRule(key=21, options=enums.SortRuleFlags(sort_descending=True))])),
            limit=10)

        # Service stubs run on the client's channel, which adds the session to each call.
        search = client.stub(IRPCSearch_pb2_grpc.IRPCSearchStub)
        response = search.SearchObjects(request)

        for item in response.results:
            version = item.object.version_info
            print(f"{version.title} (ID {item.object.object_info.obj_id.item_id.internal_id}, "
                  f"version {version.version.internal_version}, "
                  f"modified {version.last_modified_at_utc.ToDatetime():%Y-%m-%d %H:%M:%S} UTC)")


if __name__ == "__main__":
    run(main)

Source: python/quickstart.py

• MFilesClient.from_env opens the channel and logs in. Leaving the with block logs out and closes the channel.
• Every service has a generated stub, created on the client's channel with client.stub(...).
• SearchObjects takes exactly one SearchConditionArray of SearchCondition items, which must all match. Without the "not deleted" condition, users who may see deleted objects, such as administrators, get them too.
• limit=10 with a sort order returns the first 10 of all matches in that order.
• The results are SearchResultsItem messages; object holds the version information (version_info) and the object information (object_info).
• A failed call raises MFilesError, which names the M-Files error code.

is_object_type and is_not_deleted are small builders from mfiles_values.py. They show the message shapes of the two conditions; lookups to object types use the value list ID -11:

def property_condition(property_def: int, condition_type: int,
                       value: structs.TypedValue) -> structs.SearchCondition:
    """A condition on a property value, e.g. property 100 (Class) equals a lookup."""
    return structs.SearchCondition(
        expression=structs.Expression(
            type=enums.EXPRESSION_TYPE_PROPERTY_VALUE,
            data=structs.ExpressionUnion(property_value=structs.ExpressionUnion.PropertyValueData(
                property_def=property_def))),
        type=condition_type,
        value=value)


def status_condition(status: int, condition_type: int,
                     value: structs.TypedValue) -> structs.SearchCondition:
    """A condition on a status value, e.g. the object type or the deleted flag."""
    return structs.SearchCondition(
        expression=structs.Expression(
            type=enums.EXPRESSION_TYPE_STATUS_VALUE,
            data=structs.ExpressionUnion(status_value=structs.ExpressionUnion.StatusValueData(
                type=status))),
        type=condition_type,
        value=value)


def is_object_type(object_type: int) -> structs.SearchCondition:
    """Objects of one object type. Value list -11 lists the object types."""
    return status_condition(enums.STATUS_TYPE_OBJECT_TYPE, enums.CONDITION_TYPE_EQUAL,
                            lookup(-11, object_type))


def is_not_deleted() -> structs.SearchCondition:
    """Objects that are not deleted."""
    return status_condition(enums.STATUS_TYPE_IS_DELETED, enums.CONDITION_TYPE_EQUAL,
                            boolean(False))

Source: python/mfiles_values.py

More about conditions: Search.

What the helper does for you

mfiles_client.py hides the parts that the proto files cannot express.

It opens the channel, with TLS unless MFILES_TLS is false, and sets the message size limits that M-Files' own client uses: responses of any size, requests up to 48 MiB (the server's default limit):

options = [("grpc.max_receive_message_length", MAX_RECEIVE_MESSAGE_BYTES),
           ("grpc.max_send_message_length", MAX_SEND_MESSAGE_BYTES)]
if config.tls:
    root_certificates = Path(config.ca_cert).read_bytes() if config.ca_cert else None
    credentials = grpc.ssl_channel_credentials(root_certificates=root_certificates)
    self._channel = grpc.secure_channel(config.address, credentials, options=options)
else:
    self._channel = grpc.insecure_channel(config.address, options=options)
self.channel = grpc.intercept_channel(self._channel, _MetadataInterceptor(self))

Source: python/mfiles_client.py

It logs in with LogIn. The response's session_data.session_id (48 bytes) identifies the session:

request = IRPCLogin_pb2.LogInRequest(
    environment_data=structs_pb2.EnvironmentData(
        client_name=self.client_name, host_platform="Python"),
    client_data=self._client_data(),
    login_data=structs_pb2.LoginData(
        server_hostname=self.config.host, vault_guid=self.config.vault),
    authentication_data=authentication)
response = self.login.LogIn(request)
if not response.session_data.session_id:
    raise RuntimeError("LogIn did not complete: the authentication needs more steps")
# From now on, the interceptor sends this in "mfiles-session-id-bin" on every call.
self.session_id = response.session_data.session_id

Source: python/mfiles_client.py

With a user name and password, authentication_data is:

authentication = structs_pb2.AuthDataClient(
    type=enums_pb2.AUTH_DATA_TYPE_CREDENTIALS,
    data=structs_pb2.AuthDataClientUnion(credentials=structs_pb2.Credentials(
        type=enums_pb2.CREDENTIALS_TYPE_MFILES, username=username, password=password)))

Source: python/mfiles_client.py

Its interceptor adds call metadata to every call: mfiles-is-remote-call: true, a new random 16-byte mfiles-activity-id-bin, and after login the session ID as mfiles-session-id-bin. None of the request messages has a field for the session. A call that needs a session and does not have one fails with UNAUTHENTICATED. M-Files Cloud sets the cookie mfilesmsm to keep a client on one server instance; gRPC does not keep cookies, so the interceptor stores set-cookie values from the response metadata and sends them back in a cookie header on every later call. See Call metadata.

When a call fails, the interceptor raises MFilesError, decoded from the mfiles-error-serialized-bin and mfiles-error-text-bin trailers: the gRPC status (status), the message (message), the M-Files error code of the top-level error (hresult; the last entry of the error stack that is not the generic 0x80040001), every code of the stack (codes) and has(code):

self.long_message: str = trailers.get(ERROR_TEXT_KEY, b"").decode("utf-8", "replace")
self.stack = list(structs_pb2.RPCErrorInfoArray.FromString(
    trailers.get(ERROR_STACK_KEY, b"")).value)
self.codes = [entry.error_code & 0xFFFFFFFF for entry in self.stack]  # cause first
# The top-level error, skipping entries that only pass the error on.
self.hresult: Optional[int] = next(
    (code for code in reversed(self.codes) if code != E_PASS_THROUGH),
    self.codes[-1] if self.codes else None)

Source: python/mfiles_client.py

Catch it like this:

try:
    objects.GetObjectInfoAndProperties(request)
except MFilesError as error:
    if not error.has(0x8004000B):  # not found
        raise
    print(f"{error.hresult_hex}: {error.message}")

Errors lists the common codes and what to do about each.

At the end, close() calls LogOut and closes the channel.

If it does not work

Next steps

• Connect: versions, capabilities, sessions and keep-alive.
• Vault structure: find the IDs of object types, classes and properties.
• Search, Read an object, Download a file.
• Create a document, Update properties, Change the workflow state.
• Page through results and Handle errors.