Go quickstart

View as Markdown

In about 15 minutes you generate Go code from the M-Files protos, configure a connection and run a program that logs in to a vault, lists its 10 most recently modified documents and logs out. The program uses the official google.golang.org/grpc and google.golang.org/protobuf modules and mfilesclient, a small connection helper from the samples that you can copy into your own project.

Prerequisites

• Go 1.26 or later: google.golang.org/grpc v1.85.0 requires it.

• protoc, the protocol buffer compiler. Python's grpcio-tools package includes one (pip install grpcio-tools); the generate scripts use it when protoc is not on PATH.

• The Go plugins for protoc:

  go install google.golang.org/protobuf/cmd/protoc-gen-go@v1.36.12
  go install google.golang.org/grpc/cmd/protoc-gen-go-grpc@v1.6.2
  

• A vault you can reach over gRPC, and its GUID with braces ({7BBEA0BC-40D8-4BCB-A67D-8A3ACFBDB575}). Don't know the GUID? Find the vault by its name lists the vaults you can log in to, or ask your M-Files administrator. The endpoint:

  • M-Files Cloud: the vault's host name; the endpoint is <vault host>:443 with TLS.
  • On-premises: the server's host name and gRPC port, 7766 by default; plain HTTP/2 unless the administrator has enabled TLS.

• An M-Files user name and password, or a token from the vault's identity provider.

Cloud and on-premises and Connecting to the server explain the endpoints in detail.

Get the samples and the protos

Download the gRPC API bundle and unpack it. It contains:

• proto/: the API definition, one .proto file per service plus mfiles/structs.proto, mfiles/enums.proto and mfiles/api/annotations.proto, with the documentation as comments;
• samples/go/: the Go module this guide uses.

An M-Files Server installation also has the API definition, in <installation folder>\Common\Web\GRPC\proto, as one combined legacy file without documentation. It describes the same wire contract, but it does not produce the Go package path these samples import. Prefer the bundle.

Generate the code

cd samples\go
.\generate.ps1          # PowerShell; in bash: ./generate.sh

The script runs protoc with protoc-gen-go and protoc-gen-go-grpc on every file of the bundle and writes the code into gen/. It finds the bundle at ../../proto; pass another folder with -ProtoDir (or as the first argument of generate.sh), or set MFILES_PROTO_DIR.

The bundle's go_package option puts all services, messages and enums into one package, github.com/M-Files/mfiles-grpc-api/go/mfiles. The script makes gen/ a module of its own with that path, and the samples' go.mod points to it:

// The generated code: a module of its own in ./gen, written by generate.ps1 or generate.sh.
replace github.com/M-Files/mfiles-grpc-api => ./gen

Source: go/go.mod

Your code imports the package as "github.com/M-Files/mfiles-grpc-api/go/mfiles". The package is large: the first build takes a while, later builds come from the cache.

To start a project of your own, copy gen/, the replace directive and the mfilesclient folder, and require google.golang.org/grpc and google.golang.org/protobuf in your go.mod.

Configure

The samples read their settings from environment variables:

M-Files Cloud (PowerShell):

$env:MFILES_ADDRESS = 'contoso.cloudvault.m-files.com:443'
$env:MFILES_VAULT = '{7BBEA0BC-40D8-4BCB-A67D-8A3ACFBDB575}'
$env:MFILES_USERNAME = 'alice'
$env:MFILES_PASSWORD = Read-Host -Prompt 'Password'

On-premises without TLS (bash):

export MFILES_ADDRESS=mfserver:7766 MFILES_TLS=false
export MFILES_VAULT='{7BBEA0BC-40D8-4BCB-A67D-8A3ACFBDB575}'
export MFILES_USERNAME=alice
read -rs MFILES_PASSWORD && export MFILES_PASSWORD

Use MFILES_TLS=false only for a server that has no certificate, such as a lab installation.

Single sign-on. A vault that signs users in through M-Files Login Service, as is typical on M-Files Cloud, refuses a password login with "Authentication failed.". Set MFILES_TOKEN to an access token from the Login Service, and MFILES_AUTH_PLUGIN and MFILES_AUTH_SCOPE to the plugin and scope that the vault returns. M-Files Login Service shows how to get the token; the Python sample recipes/login_service.py does it end to end. Getting an SSO token is up to your application; see Authentication and sessions.

Run

go run ./quickstart

The output lists the documents, newest first. In the M-Files Sample Vault it starts like this:

Project Plan (template) (ID 445, version 1, modified 2012-06-07 12:01:57 UTC)
Project Tasks - CRM Development (ID 444, version 1, modified 2012-06-07 11:54:29 UTC)
Corporate Presentation (ID 443, version 1, modified 2012-06-07 11:40:14 UTC)
...

The code

// Connect and log in with the settings in the MFILES_* environment variables.
cfg, err := mfilesclient.ConfigFromEnv()
if err != nil {
	return err
}
cfg.ClientName = "M-Files gRPC quickstart"
client, err := mfilesclient.Connect(ctx, cfg)
if err != nil {
	return err
}
defer client.Close() // logs out and closes the channel

// Documents (object type 0) that are not deleted, sorted by "Last modified" (property 21),
// newest first. The server sorts before it applies the limit.
search := mfiles.NewIRPCSearchClient(client.Conn)
response, err := search.SearchObjects(ctx, &mfiles.SearchObjectsRequest{
	Conditions: mfilesclient.AllOf(mfilesclient.IsObjectType(0), mfilesclient.IsNotDeleted()),
	Sorting:    mfilesclient.SortBy(21, true),
	Limit:      10,
})
if err != nil {
	return err
}

for _, item := range response.GetResults() {
	object := item.GetObject()
	version := object.GetVersionInfo()
	fmt.Printf("%s (ID %d, version %d, modified %s UTC)\n",
		version.GetTitle(),
		object.GetObjectInfo().GetObjId().GetItemId().GetInternalId(),
		version.GetVersion().GetInternalVersion(),
		version.GetLastModifiedAtUtc().AsTime().Format(time.DateTime))
}

Source: go/quickstart/main.go

• mfilesclient.Connect creates the channel and logs in. client.Close logs out and closes the channel.
• Every service has a generated client, created on the channel: mfiles.NewIRPCSearchClient(client.Conn).
• The search has one array of conditions; all conditions in an array must match. Without the "not deleted" condition, users who may see deleted objects get them too.
• Limit: 10 with a sort order returns the first 10 of all matches in that order.
• A failed call returns an *mfilesclient.Error that names the M-Files error code.

IsObjectType and IsNotDeleted are small builders from the helper. They show the message shapes of the two conditions:

// StatusCondition compares a status value of the object, such as its object type or whether
// it is deleted.
func StatusCondition(status mfiles.StatusType, condition mfiles.ConditionType,
	value *mfiles.TypedValue) *mfiles.SearchCondition {
	return &mfiles.SearchCondition{
		Expression: &mfiles.Expression{
			Type: mfiles.ExpressionType_EXPRESSION_TYPE_STATUS_VALUE,
			Data: &mfiles.ExpressionUnion{Data: &mfiles.ExpressionUnion_StatusValue{
				StatusValue: &mfiles.ExpressionUnion_StatusValueData{Type: status},
			}},
		},
		Type:  condition,
		Value: value,
	}
}

// PropertyCondition compares a property value of the object, such as its class (property 100)
// or "Last modified" (property 21).
func PropertyCondition(propertyDef int32, condition mfiles.ConditionType,
	value *mfiles.TypedValue) *mfiles.SearchCondition {
	return &mfiles.SearchCondition{
		Expression: &mfiles.Expression{
			Type: mfiles.ExpressionType_EXPRESSION_TYPE_PROPERTY_VALUE,
			Data: &mfiles.ExpressionUnion{Data: &mfiles.ExpressionUnion_PropertyValue{
				PropertyValue: &mfiles.ExpressionUnion_PropertyValueData{PropertyDef: propertyDef},
			}},
		},
		Type:  condition,
		Value: value,
	}
}

// IsObjectType matches objects of one object type (0 = document). Value list -11 lists the
// object types.
func IsObjectType(objectType int32) *mfiles.SearchCondition {
	return StatusCondition(mfiles.StatusType_STATUS_TYPE_OBJECT_TYPE,
		mfiles.ConditionType_CONDITION_TYPE_EQUAL, Lookup(-11, objectType))
}

// IsNotDeleted matches objects that are not deleted.
func IsNotDeleted() *mfiles.SearchCondition {
	return StatusCondition(mfiles.StatusType_STATUS_TYPE_IS_DELETED,
		mfiles.ConditionType_CONDITION_TYPE_EQUAL, Boolean(false))
}

Source: go/mfilesclient/search.go

Go and the generated code

• Enum values carry their type's name: DATATYPE_TEXT of the enum Datatype is mfiles.Datatype_DATATYPE_TEXT.
• Each member of a oneof is a wrapper type. A text value is &mfiles.TypedValueUnion{Data: &mfiles.TypedValueUnion_Text{Text: "..."}}. Read one with a type switch on GetData().
• Use the getters (GetObject(), GetVersionInfo(), ...) to read responses: they return zero values instead of panicking when a message field is unset.

The helper has constructors for the common values:

// Text is a single-line text value.
func Text(value string) *mfiles.TypedValue {
	return &mfiles.TypedValue{
		Type: mfiles.Datatype_DATATYPE_TEXT,
		Data: &mfiles.TypedValueUnion{Data: &mfiles.TypedValueUnion_Text{Text: value}},
	}
}

// Integer is an integer value.
func Integer(value int32) *mfiles.TypedValue {
	return &mfiles.TypedValue{
		Type: mfiles.Datatype_DATATYPE_INTEGER,
		Data: &mfiles.TypedValueUnion{Data: &mfiles.TypedValueUnion_Integer{Integer: value}},
	}
}

// Boolean is a yes/no value.
func Boolean(value bool) *mfiles.TypedValue {
	return &mfiles.TypedValue{
		Type: mfiles.Datatype_DATATYPE_BOOLEAN,
		Data: &mfiles.TypedValueUnion{Data: &mfiles.TypedValueUnion_Boolean{Boolean: value}},
	}
}

// Timestamp is a date and time value, such as "Last modified" (property 21).
func Timestamp(value time.Time) *mfiles.TypedValue {
	return &mfiles.TypedValue{
		Type: mfiles.Datatype_DATATYPE_TIMESTAMP,
		Data: &mfiles.TypedValueUnion{Data: &mfiles.TypedValueUnion_Timestamp{
			Timestamp: timestamppb.New(value),
		}},
	}
}

// Lookup is a single-select lookup value: the item with the given ID in a value list (or an
// object of the object type with that ID).
func Lookup(valueList, item int32) *mfiles.TypedValue {
	return &mfiles.TypedValue{
		Type: mfiles.Datatype_DATATYPE_LOOKUP,
		Data: &mfiles.TypedValueUnion{Data: &mfiles.TypedValueUnion_Lookup{
			Lookup: lookupItem(valueList, item),
		}},
	}
}

// MultiSelectLookup is a multi-select lookup value.
func MultiSelectLookup(valueList int32, items ...int32) *mfiles.TypedValue {
	lookups := make([]*mfiles.Lookup, 0, len(items))
	for _, item := range items {
		lookups = append(lookups, lookupItem(valueList, item))
	}
	return &mfiles.TypedValue{
		Type: mfiles.Datatype_DATATYPE_MULTI_SELECT_LOOKUP,
		Data: &mfiles.TypedValueUnion{Data: &mfiles.TypedValueUnion_MultiSelectLookup{
			MultiSelectLookup: &mfiles.MultiSelectLookup{Values: lookups},
		}},
	}
}

func lookupItem(valueList, item int32) *mfiles.Lookup {
	return &mfiles.Lookup{
		ValueListItemInfo: &mfiles.ItemInfo{ObjId: ObjID(valueList, item)},
		Version: &mfiles.ObjVerVersion{
			Type:            mfiles.ObjVerVersionType_OBJ_VER_VERSION_TYPE_LATEST,
			InternalVersion: -1,
		},
	}
}

Source: go/mfilesclient/values.go

What the helper does

gRPC alone is not enough to talk to M-Files: part of the contract travels in call metadata, and errors arrive in trailers. mfilesclient takes care of that.

The channel. TLS with the system's root certificates plus an optional PEM file, or no TLS at all, and the message size limits that M-Files' own client uses: responses of any size instead of gRPC's default 4 MiB, requests up to 48 MiB (the server's default limit). Two interceptors handle every call:

// Dial creates the channel to cfg.Address. It does not log in and does not contact the server
// yet: the first call does.
func Dial(cfg Config) (*Client, error) {
	transport, err := transportCredentials(cfg)
	if err != nil {
		return nil, err
	}
	client := &Client{host: cfg.host()}
	client.Conn, err = grpc.NewClient(cfg.Address,
		grpc.WithTransportCredentials(transport),
		grpc.WithDefaultCallOptions(
			grpc.MaxCallRecvMsgSize(MaxReceiveMessageSize),
			grpc.MaxCallSendMsgSize(MaxSendMessageSize)),
		// Every call carries the M-Files metadata; failed calls return *Error.
		grpc.WithChainUnaryInterceptor(client.unaryInterceptor),
		grpc.WithChainStreamInterceptor(client.streamInterceptor))
	if err != nil {
		return nil, err
	}
	return client, nil
}

// transportCredentials is TLS with the system's root certificates, plus cfg.CACertFile when set;
// or no TLS at all when cfg.TLS is false.
func transportCredentials(cfg Config) (credentials.TransportCredentials, error) {
	if !cfg.TLS {
		return insecure.NewCredentials(), nil
	}
	roots, err := x509.SystemCertPool()
	if err != nil {
		roots = x509.NewCertPool()
	}
	if cfg.CACertFile != "" {
		pem, err := os.ReadFile(cfg.CACertFile)
		if err != nil {
			return nil, err
		}
		if !roots.AppendCertsFromPEM(pem) {
			return nil, fmt.Errorf("%s: no PEM certificates found", cfg.CACertFile)
		}
	}
	return credentials.NewTLS(&tls.Config{RootCAs: roots, MinVersion: tls.VersionTLS12}), nil
}

Source: go/mfilesclient/client.go

Call metadata. Every call carries mfiles-is-remote-call: true and a new random 16-byte mfiles-activity-id-bin; after login also the session ID in mfiles-session-id-bin. In Go the value of a -bin key is the raw bytes in a string; gRPC base64-encodes it on the wire. M-Files Cloud sets the cookie mfilesmsm to keep a client on one server instance; gRPC does not keep cookies, so the interceptor stores set-cookie values and sends them back in a cookie header (see Call metadata):

// withMetadata adds the M-Files call metadata to an outgoing call: the remote-call flag, a new
// activity ID, the session ID once logged in, and the cookies the server has set.
func (c *Client) withMetadata(ctx context.Context) context.Context {
	activityID := make([]byte, 16)
	_, _ = rand.Read(activityID)
	pairs := []string{headerRemoteCall, "true", headerActivityID, string(activityID)}
	if session := c.sessionID(); len(session) > 0 {
		pairs = append(pairs, headerSessionID, string(session))
	}
	if cookies := c.cookieHeader(); cookies != "" {
		pairs = append(pairs, headerCookie, cookies)
	}
	return metadata.AppendToOutgoingContext(ctx, pairs...)
}

// unaryInterceptor adds the metadata to every unary call, keeps the cookies and the metadata
// version the server returns, and turns a failure into *Error.
func (c *Client) unaryInterceptor(ctx context.Context, method string, req, reply any,
	cc *grpc.ClientConn, invoker grpc.UnaryInvoker, opts ...grpc.CallOption) error {
	var header, trailer metadata.MD
	opts = append(opts, grpc.Header(&header), grpc.Trailer(&trailer))
	err := invoker(c.withMetadata(ctx), method, req, reply, cc, opts...)
	c.readResponseMetadata(header, trailer)
	return decodeError(err, trailer)
}

Source: go/mfilesclient/metadata.go

Login. LogIn binds the session to the vault in login_data.vault_guid. Its response's session_data.session_id (48 bytes) goes into mfiles-session-id-bin from then on:

// LogInWith logs in to a vault. From then on every call carries the session.
func (c *Client) LogInWith(ctx context.Context, vault string, auth *mfiles.AuthDataClient,
	clientName string) (*mfiles.SessionData, error) {
	if clientName == "" {
		clientName = filepath.Base(os.Args[0])
	}
	response, err := mfiles.NewIRPCLoginClient(c.Conn).LogIn(ctx, &mfiles.LogInRequest{
		EnvironmentData: &mfiles.EnvironmentData{ClientName: clientName, HostPlatform: "Go"},
		ClientData: &mfiles.ClientData{
			Type:     mfiles.ClientType_CLIENT_TYPE_SERVER_API,
			Language: "en",
			// A plugin (token) login uses the 32-byte form of the attempt identifier.
			Capabilities: &mfiles.ClientCapabilities{LongAuthenticationAttemptIdentifier: true},
		},
		LoginData:          &mfiles.LoginData{ServerHostname: c.host, VaultGuid: vault},
		AuthenticationData: auth,
	})
	if err != nil {
		return nil, err
	}
	session := response.GetSessionData()
	if len(session.GetSessionId()) == 0 {
		// A plugin that needs more than one round trip answers without a session.
		return nil, errors.New("LogIn returned no session: multi-step authentication is not supported")
	}
	// From now on the interceptor sends session_id in mfiles-session-id-bin on every call.
	c.mu.Lock()
	c.session = session
	c.mu.Unlock()
	return session, nil
}

Source: go/mfilesclient/client.go

With a user name and password, authentication_data is:

// PasswordAuth is the authentication data of an M-Files user name and password.
func PasswordAuth(username, password string) *mfiles.AuthDataClient {
	return &mfiles.AuthDataClient{
		Type: mfiles.AuthDataType_AUTH_DATA_TYPE_CREDENTIALS,
		Data: &mfiles.AuthDataClientUnion{Data: &mfiles.AuthDataClientUnion_Credentials{
			Credentials: &mfiles.Credentials{
				Type:     mfiles.CredentialsType_CREDENTIALS_TYPE_MFILES,
				Username: username,
				Password: password,
			},
		}},
	}
}

Source: go/mfilesclient/client.go

Errors. A failed call returns *mfilesclient.Error, decoded from the mfiles-error-serialized-bin and mfiles-error-text-bin trailers: the gRPC status (Status), the message, the M-Files error code of the top-level error (HResult, such as 0x8004000B "Not found."; the last entry of the error stack that is not the generic 0x80040001), every code of the stack (Codes) and Has(code). It keeps the gRPC status, so status.Code(err) still works:

// Error is a failed M-Files call. Test for it with errors.As; status.Code(err) still gives the
// gRPC status code.
type Error struct {
	Status  *status.Status         // the gRPC status: Unknown for M-Files errors
	HResult uint32                 // the M-Files error code, e.g. 0x8004000B; 0 when unknown
	Codes   []uint32               // the codes of the error stack, the original error first
	Message string                 // the error message (the gRPC status message)
	Details string                 // the long error message with the server's error stack
	Stack   []*mfiles.RPCErrorInfo // the server's error stack, the original error first
}

func (e *Error) Error() string {
	if len(e.Codes) == 0 {
		return fmt.Sprintf("gRPC error %s: %s", e.Status.Code(), e.Message)
	}
	return fmt.Sprintf("M-Files error 0x%08X (%s): %s", e.HResult, e.Status.Code(), e.Message)
}

// SignedHResult is the error code as a signed 32-bit integer, e.g. -2147221493.
func (e *Error) SignedHResult() int32 { return int32(e.HResult) }

// GRPCStatus lets status.Code and status.FromError see through the *Error.
func (e *Error) GRPCStatus() *status.Status { return e.Status }

// Has reports whether code is anywhere in the error stack.
func (e *Error) Has(code uint32) bool {
	for _, c := range e.Codes {
		if c == code {
			return true
		}
	}
	return false
}

// AsError returns the *Error in err's chain, if there is one.
func AsError(err error) (*Error, bool) {
	var mfilesErr *Error
	ok := errors.As(err, &mfilesErr)
	return mfilesErr, ok
}

// decodeError turns a failed call into *Error, using the error trailers when present.
func decodeError(err error, trailer metadata.MD) error {
	if err == nil {
		return nil
	}
	st, ok := status.FromError(err)
	if !ok || st.Code() == codes.OK {
		return err
	}
	result := &Error{Status: st, Message: st.Message()}
	if values := trailer.Get(headerErrorText); len(values) > 0 {
		result.Details = values[0]
	}
	if values := trailer.Get(headerErrorStack); len(values) > 0 {
		stack := &mfiles.RPCErrorInfoArray{}
		if proto.Unmarshal([]byte(values[0]), stack) == nil {
			result.Stack = stack.GetValue()
		}
	}
	for _, info := range result.Stack {
		result.Codes = append(result.Codes, uint32(info.GetErrorCode()))
	}
	result.HResult = errorCode(result.Codes)
	return result
}

// errorPassThrough is the generic code of an error that is only passed up the stack; it does
// not say what went wrong.
const errorPassThrough = 0x80040001

// errorCode is the code of the top-level error, the last entry of the stack: the error the
// status message describes, such as 0x800407E4 for "Access denied." Entries with the generic
// code 0x80040001 are skipped.
func errorCode(stack []uint32) uint32 {
	for i := len(stack) - 1; i >= 0; i-- {
		if stack[i] != errorPassThrough {
			return stack[i]
		}
	}
	if len(stack) > 0 {
		return stack[len(stack)-1]
	}
	return 0
}

Source: go/mfilesclient/errors.go

Handle it like this:

_, err := objects.GetObjectInfoAndProperties(ctx, request)
if mfilesErr, ok := mfilesclient.AsError(err); ok && mfilesErr.Has(mfilesclient.CodeNotFound) {
	fmt.Printf("0x%08X: %s\n", mfilesErr.HResult, mfilesErr.Message)
}

Errors lists the common codes and what to do about each.

The metadata structure version. Calls that report the vault's metadata structure version return it in the mfiles-metadata-version-bin trailer; client.MetadataStructureVersion() has the latest one. See Metadata structure version and caching.

Troubleshooting

go run ./recipes/connect calls QueryVersion before it logs in, so it tells connection problems from login problems.

Next steps

• Run the task recipes: go run ./recipes/<name>, with connect, vault_structure, search, read_object, download_file, create_document, update_properties, workflow_state, paging or errors. The samples' README.md lists their arguments; run-tests.ps1 runs them all.
• The recipes explained: Connect, Vault structure, Search, Read an object, Download a file, Create a document, Update properties, Change the workflow state, Page through results and Handle errors.
• Learn the model: Vault and metadata structure, Objects and versions, Properties and values and Search.
• Keep a session healthy in a long-running program: Authentication and sessions.
• Find any method in the reference, starting with SearchObjects and GetObjectInfoAndProperties.