Azure App registration

JSON

private.json sample:

{
	"siteUrl": "https://contoso.sharepoint.com/sites/test",
	"tenantId": "e4d43069-8ecb-49c4-8178-5bec83c53e9d",
	"clientId": "628cc712-c9a4-48f0-a059-af64bdbb4be5",
	"username": "user@contoso.com",
	"password": "password"
}

Usage sample

package main

import (
	"fmt"
	"log"
	"os"

	"github.com/koltyakov/gosip"
	"github.com/koltyakov/gosip/api"
	strategy "github.com/koltyakov/gosip/auth/azurecreds"
)

func main() {

	// authCnfg := &strategy.AuthCnfg{
	// 	SiteURL:  os.Getenv("SPAUTH_SITEURL"),
	// 	TenantID: os.Getenv("AZURE_TENANT_ID"),
	// 	ClientID: os.Getenv("AZURE_CLIENT_ID"),
	// 	Username: os.Getenv("AZURE_USERNAME"),
	// 	Password: os.Getenv("AZURE_PASSWORD"),
	// }
	// or using `private.json` creds source

	authCnfg := &strategy.AuthCnfg{}
	configPath := "./config/private.json"
	if err := authCnfg.ReadConfig(configPath); err != nil {
		log.Fatalf("unable to get config: %v", err)
	}
	
	client := &gosip.SPClient{AuthCnfg: authCnfg}
	sp := api.NewSP(client)

	res, err := sp.Web().Select("Title").Get()
	if err != nil {
		log.Fatal(err)
	}

	fmt.Printf("Site title: %s\n", res.Data().Title)

}

Main features

• Unattended authentication using different strategies.

• Simplified API consumption (REST, CSOM, SOAP, etc.).

• SharePoint-aware embedded features (retries, header presets, error handling).

Supported SharePoint versions

• SharePoint Online (SPO).

• On-Premises (2019/2016/2013).

Supported auth strategies

• SharePoint Online:
• SharePoint On-Premises 2019/2016/2013:

Installation

go get github.com/koltyakov/gosip

Usage insights

Understand SharePoint environment type and authentication strategy

Let's assume it's SharePoint Online and Add-In Only permissions. Then strategy "github.com/koltyakov/gosip/auth/addin" sub package should be used.

package main

import (
  "github.com/koltyakov/gosip"
  "github.com/koltyakov/gosip/api"
  strategy "github.com/koltyakov/gosip/auth/addin"
)

Initiate authentication object

auth := &strategy.AuthCnfg{
  SiteURL:      os.Getenv("SPAUTH_SITEURL"),
  ClientID:     os.Getenv("SPAUTH_CLIENTID"),
  ClientSecret: os.Getenv("SPAUTH_CLIENTSECRET"),
}

AuthCnfg's from different strategies contains different options relevant for a specified auth type.

configPath := "./config/private.json"
auth := &strategy.AuthCnfg{}

err := auth.ReadConfig(configPath)
if err != nil {
  fmt.Printf("Unable to get config: %v\n", err)
  return
}

Bind auth client with Fluent API

client := &gosip.SPClient{AuthCnfg: auth}

sp := api.NewSP(client)

res, err := sp.Web().Select("Title").Get()
if err != nil {
  fmt.Println(err)
}

fmt.Printf("%s\n", res.Data().Title)

Usage samples

Fluent API client

Provides a simple way of constructing API endpoint calls with IntelliSense and chainable syntax.

package main

import (
  "encoding/json"
  "fmt"
  "log"

  "github.com/koltyakov/gosip"
  "github.com/koltyakov/gosip/api"
  strategy "github.com/koltyakov/gosip/auth/addin"
)

func main() {
  // Getting auth params and client
  client, err := getAuthClient()
  if err != nil {
    log.Fatalln(err)
  }

  // Binding SharePoint API
  sp := api.NewSP(client)

  // Custom headers
  headers := map[string]string{
    "Accept": "application/json;odata=minimalmetadata",
    "Accept-Language": "de-DE,de;q=0.9",
  }
  config := &api.RequestConfig{Headers: headers}

  // Chainable request sample
  data, err := sp.Conf(config).Web().Lists().Select("Id,Title").Get()
  if err != nil {
    log.Fatalln(err)
  }

  // Response object unmarshalling
  // (struct depends on OData mode and API method)
  res := &struct {
    Value []struct {
      ID    string `json:"Id"`
      Title string `json:"Title"`
    } `json:"value"`
  }{}

  if err := json.Unmarshal(data, &res); err != nil {
    log.Fatalf("unable to parse the response: %v", err)
  }

  for _, list := range res.Value {
    fmt.Printf("%+v\n", list)
  }

}

func getAuthClient() (*gosip.SPClient, error) {
  configPath := "./config/private.spo-addin.json" // <- file with creds
  auth := &strategy.AuthCnfg{}
  if err := auth.ReadConfig(configPath); err != nil {
    return nil, fmt.Errorf("unable to get config: %v", err)
  }
  return &gosip.SPClient{AuthCnfg: auth}, nil
}

Generic HTTP-client helper

Provides generic GET/POST helpers for REST operations, reducing amount of http.NewRequest scaffolded code, can be used for custom or not covered with a Fluent API endpoints.

package main

import (
  "fmt"
  "log"

  "github.com/koltyakov/gosip"
  "github.com/koltyakov/gosip/api"
  strategy "github.com/koltyakov/gosip/auth/ntlm"
)

func main() {
  configPath := "./config/private.ntlm.json"
  auth := &strategy.AuthCnfg{}

  if err := auth.ReadConfig(configPath); err != nil {
    log.Fatalf("unable to get config: %v\n", err)
  }

  sp := api.NewHTTPClient(&gosip.SPClient{AuthCnfg: auth})

  endpoint := auth.GetSiteURL() + "/_api/web?$select=Title"

  data, err := sp.Get(endpoint, nil)
  if err != nil {
    log.Fatalf("%v\n", err)
  }

  // sp.Post(endpoint, []byte(body), nil) // generic POST

  // generic DELETE helper crafts "X-Http-Method"="DELETE" header
  // sp.Delete(endpoint, nil)

  // generic UPDATE helper crafts "X-Http-Method"="MERGE" header
  // sp.Update(endpoint, nil)

  // CSOM helper (client.svc/ProcessQuery)
  // sp.ProcessQuery(endpoint, []byte(body))

  fmt.Printf("response: %s\n", data)
}

Low-level HTTP-client usage

Low-lever SharePoint-aware HTTP client from github.com/koltyakov/gosip package for custom or not covered with a Fluent API client endpoints with granular control for HTTP request, response, and http.Client parameters. Used internally but almost never required in a consumer code.

client := &gosip.SPClient{AuthCnfg: auth}

var req *http.Request
// Initiate API request
// ...

resp, err := client.Execute(req)
if err != nil {
  fmt.Printf("Unable to request api: %v", err)
  return
}

SPClient has Execute method which is a wrapper function injecting SharePoint authentication and ending up calling http.Client's Do method.

Reference

License

Supported strategies

Honorable mentions

Why Golang?

Go has many strong parts. It's damn simple, friendly for large team projects, super fast in a compilation, fast in runtime and requires little or no prerequisites. It is friendly for concurrent processes, modern hardware architectures, network applications and cloud-based solutions, DevOps tool.

Go promises backward compatibility and long support period. Many say that Go is "the next enterprise big thing" and one of the default preferred languages. Lots of modern huge and awesome applications are written in Go while staying simple and reliable. Many main vendors use Go in one way or another.

For whom is this for?

In the first place, Gosip is for developers who use Go on their daily basis and who got to integrate their apps with SharePoint without diving too deep to learn it from the grouds up.

SharePoint is complicated. We spent years solving different complex and misc problems with the platform. Not only for the end-users and customers but also for other developers and Open Source community.

Recently more and more of our internal tools have been written in Go. Our opinion is that a tool is better if it is used in a variety of different scenarios and by different teams. So we decided to deliver our Go+SP experience to Open Source and improve it together with you.

If you have no idea what is Go probably Gosip is not for you, but if you have no "Why Go?" question Gosip is an option to master SharePoint API with ease.

Where to use?

Anywhere where you considering an application written in Go reasonable.

Use-cases can vary, let me describe just a few we use the library intensively:

• Web services and APIs (dedicated web API consumed by many clients)

• Microservices applications

• Schedule runners, workflow workers, queue listeners

• CLI based tools & scripting

• Relays and gateways with elements of API communication

• Development toolchains

We would love to hear your unique use-case and even help you with our opinion.

Why not C#, PowerShell or Node.js

Under any circumstances, we're not even suggesting replacing anything with anything. It's just wrong positioning.

Together is stronger. For no reason don't search for a holly war aspect use whatever feels better for you for delivering great product or service.

Authentication strategies

Auth strategy should be selected corresponding to your SharePoint environment and its configuration.

Import path strategy "github.com/koltyakov/gosip/auth/{strategy}". Where /{strategy} stands for a strategy auth package.

JSON and struct representations are different in terms of language notations. So credentials parameters names in private.json files and declared as structs initiators vary.

Additional strategies

Secrets encoding

Azure App registration

1. Create or use existing app registration

2. Make sure that the app is configured for a specific auth scenario:

• Client credentials (might not work with SharePoint but require a Certificate-based auth)

• Certificate

• Username/Password (public clients flows must be enabled)

• Managed identity

• O365 Admin -> Azure Active Directory

• Generate self-signed certificate

# PowerShell, run on a Windows machine
$certName = "MyCert"
$password = "MyPassword"

$startDate = Get-Date
$endDate = (Get-Date).AddYears(5)
$securePass = (ConvertTo-SecureString -String $password -AsPlainText -Force)

.\Create-SelfSignedCertificate.ps1 -CommonName $certName -StartDate $startDate -EndDate $endDate -Password $securePass

or on a Linux or macOS client via openssl:

chmod +x ./Create-SelfSignedCertificate.sh
./Create-SelfSignedCertificate.sh

• New App Registration

  • Accounts in this organizational directory only

  • API Permissions -> SharePoint :: Application :: Sites.FullControl.All -> Grant Admin Consent

  • Certificates & Secrets -> Upload .cer file

• Use environment variables to provide creds bindings:

  • AZURE_TENANT_ID - Directory (tenant) ID in App Registration

  • AZURE_CLIENT_ID - Application (client) ID in App Registration

  • For certificate-base auth:

    • AZURE_CERTIFICATE_PATH - path to .pfx file

    • AZURE_CERTIFICATE_PASSWORD - password used for self-signed certificate

  • For username/password auth:

    • AZURE_USERNAME

    • AZURE_PASSWORD

Auth configuration and usage

package main

import (
    "fmt"
    "log"
    "os"

    "github.com/koltyakov/gosip"
    "github.com/koltyakov/gosip/api"
    strategy "github.com/koltyakov/gosip-sandbox/strategies/azureenv"
)

func main() {

    // os.Setenv("AZURE_TENANT_ID", "b1bacba7-c38a-414b-8c8b-65df26a15749")
    // os.Setenv("AZURE_CLIENT_ID", "8ca10ce6-c3d5-47c6-b803-0ef3b619f464")
    // os.Setenv("AZURE_CERTIFICATE_PATH", "/path/to/cert.pfx")
    // os.Setenv("AZURE_CERTIFICATE_PASSWORD", "cert-password")

    authCnfg := &strategy.AuthCnfg{
        SiteURL: os.Getenv("SPAUTH_SITEURL"),
    }

    client := &gosip.SPClient{AuthCnfg: authCnfg}
    sp := api.NewSP(client)

    res, err := sp.Web().Select("Title").Get()
    if err != nil {
        log.Fatal(err)
    }

    fmt.Printf("Site title: %s\n", res.Data().Title)

}

Environment variables auto-injection

Environment variables can be automatically injected in a runtime for Azure AAD library. To use injection add correcponding environment variables in private.json into env JSON property:

{
  "siteUrl": "https://contoso.sharepoint.com/sites/site",
  "env": {
    "AZURE_TENANT_ID": "1efde0dc-21f5-4d3d-a053-1da762c7838c",
    "AZURE_CLIENT_ID": "7278fe9b-acd5-4be5-b688-999603560d31",
    "AZURE_CERTIFICATE_PATH": "./certs/MyCert.pfx",
    "AZURE_CERTIFICATE_PASSWORD": "MyPassword"
  }
}

If you want users to sign in interactively, the best way is through device token authentication. This authentication flow passes the user a token to paste into a Microsoft sign-in site, where they then authenticate with an Azure Active Directory (AAD) account. This authentication method supports accounts that have multi-factor authentication enabled, unlike standard username/password authentication.

Azure App registration

1. Create or use existing app registration

2. Make sure that the app is configured to support device flow

• Authentication settings

  • Public client/native (mobile & desktop)
• App permissions

  • Azure Service Management :: user_impersonation

  • SharePoint :: based on your application requirements

• etc. based on application needs

Auth configuration and usage

When started the application interacts with user using device login.

After opening the link, providing device code and authenticating in browser the app is ready for communication with your SharePoint site.

The strategy caches auth token in the context of the AAD ClientID. As a result, you won't see the sign in message. If it's not the desired behavior .CleanTokenCache() method can be called to clean the local cache.

Note, that the technique is mostly applicable when user interaction is assumed. Usage of that auth approach in the headless scenarios is not the best as it can lead "stuck" application if no-one expects sign in interaction.

This authentication option uses Microsoft Online Security Token Service https://login.microsoftonline.com/extSTS.srf and SAML tokens in order to obtain authentication cookie.

Struct

JSON

private.json sample:

Code sample

Struct

Realm can be left empty or filled in, that will add small performance improvement. The easiest way to find tenant is to open SharePoint Online site collection, click Site Settings -> Site App Permissions. Taking any random app, the tenant ID (realm) is the GUID part after the @.

JSON

private.json sample:

Code sample

Extending client secrets

It's important to know that the legacy AddIn authentication's Client Secrets are issued for a limited time. After expiration, if not managed right way there is a risk to get a service connection aunothorized with the following message:

Known issues

In new subscriptions you can face Grant App Permission disabled. You'll be getting the following error:

To enable this feature, connect to SharePoint using Windows PowerShell and then run:

set-spotenant -DisableCustomAppAuthentication $false.

For AddIn Only authentication to work register new addin within your SharePoint Online tenant.

• Navigate to app registration page: https://{organization}.sharepoint.com/sites/{site}/_layouts/15/appregnew.aspx

• Click "Generate" button next to Client Id and Client Secret, fill in Title, App Domain, Redirect URI (you can type in any values you want).

• Copy Client Id and Client Secret and press "Create" button.

• Apply permissions for the app on tenant or site collection level.

Tenant scoped parmissions

https://{organization}-admin.sharepoint.com/_layouts/15/appinv.aspx

Site collection scoped permissions

https://{organization}.sharepoint.com/sites/{site}/_layouts/15/appinv.aspx

• Resolve addin by Client Id and paste in App's Permissions Request XML:

• Click "Create" and "Trust It".

To check which app principals are assigned for a site collection use:

https://{organization}.sharepoint.com/sites/{site}/_layouts/15/appprincipals.aspx

Disabled by default

In new subscriptions you could be needed to enable Grant App Permission. Connect to SharePoint using Windows PowerShell and then run:

set-spotenant -DisableCustomAppAuthentication $false.

This type of authentication uses HTTP NTLM handshake in order to obtain authentication header.

Struct

JSON

private.json sample:

or

Code sample

Azure App registration

1. Create or use existing app registration

2. Make sure that the app is configured for a specific auth scenario:

• Certificate

• O365 Admin -> Azure Active Directory

• Generate self-signed certificate

or on a Linux or macOS client via openssl:

• New App Registration

  • Accounts in this organizational directory only

  • API Permissions -> SharePoint :: Application :: Sites.FullControl.All -> Grant Admin Consent

  • Certificates & Secrets -> Upload .cer file

JSON

private.json sample:

Usage sample

Struct

type AuthCnfg struct {
  // SPSite or SPWeb URL, which is the context target for the API calls
  SiteURL string `json:"siteUrl"`
  // Username for SharePoint On-Prem,
  // format depends in FBA settings, can include domain or doesn't
  Username string `json:"username"`
  // User password
  Password string `json:"password"`
}

JSON

private.json sample:

{
  "siteUrl": "https://www.contoso.com/sites/test",
  "username": "john.doe",
  "password": "this-is-not-a-real-password"
}

Code sample

package main

import (
	"log"
	// "os"

	"github.com/koltyakov/gosip"
	strategy "github.com/koltyakov/gosip/auth/fba"
)

func main() {
	// authCnfg := &strategy.AuthCnfg{
	// 	SiteURL:  os.Getenv("SPAUTH_SITEURL"),
	// 	Username: os.Getenv("SPAUTH_USERNAME"),
	// 	Password: os.Getenv("SPAUTH_PASSWORD"),
	// }
	// or using `private.json` creds source

	authCnfg := &strategy.AuthCnfg{}
	configPath := "./config/private.json"
	if err := authCnfg.ReadConfig(configPath); err != nil {
		log.Fatalf("unable to get config: %v", err)
	}

	client := &gosip.SPClient{AuthCnfg: authCnfg}
	// use client in raw requests or bind it with Fluent API ...
}

Currently is legacy but was a popular way of exposing SharePoint into external world back in the days.

Struct

type AuthCnfg struct {
  // SPSite or SPWeb URL, which is the context target for the API calls
  SiteURL string `json:"siteUrl"`
  // Username for SharePoint On-Prem,
  // format depends in TMG settings, can include domain or doesn't
  Username string `json:"username"`
  // User password
  Password string `json:"password"`
}

JSON

private.json sample:

{
  "siteUrl": "https://www.contoso.com/sites/test",
  "username": "john.doe",
  "password": "this-is-not-a-real-password"
}

Code sample

package main

import (
	"log"
	// "os"

	"github.com/koltyakov/gosip"
	strategy "github.com/koltyakov/gosip/auth/tmg"
)

func main() {
	// authCnfg := &strategy.AuthCnfg{
	// 	SiteURL:  os.Getenv("SPAUTH_SITEURL"),
	// 	Username: os.Getenv("SPAUTH_USERNAME"),
	// 	Password: os.Getenv("SPAUTH_PASSWORD"),
	// }
	// or using `private.json` creds source
	
	authCnfg := &strategy.AuthCnfg{}
	configPath := "./config/private.json"
	if err := authCnfg.ReadConfig(configPath); err != nil {
		log.Fatalf("unable to get config: %v", err)
	}

	client := &gosip.SPClient{AuthCnfg: authCnfg}
	// use client in raw requests or bind it with Fluent API ...
}

When you deal with multiple SharePoint environments with different strategies and concidering same code automatically resolving a strategy use the dynamic auth helper.

With the dynamic auth you'd need extending a private.json content with strategy property, containing a name of a strategy. E.g.:

{
  "strtegy": "addin",
  "siteUrl": "https://contoso.sharepoint.com/sites/site",
  "clientId": "...",
  "clientSecret": "..."
}

Available strategy names are: azurecert, azurecreds, device, addin, adfs, fba, ntlm, saml, tmg.

Usage

package main

import (
	"flag"
	"log"

	"github.com/koltyakov/gosip"
	"github.com/koltyakov/gosip/api"
	"github.com/koltyakov/gosip/auth"
)

func main() {
	config := flag.String("config", "./config/private.json", "Config path")

	flag.Parse()

	authCnfg, err := auth.NewAuthFromFile(*config)
	if err != nil {
		log.Fatalf("unable to get config: %v", err)
	}
	
	// or
	// authCnfg, _ := NewAuthFromFile(strategyName)
	// _ = auth.ParseConfig(byteJsonCreds)

	client := &gosip.SPClient{AuthCnfg: authCnfg}
	sp := api.NewSP(client)

	// ...

}

During the development, it's common to face a situation when production-level auth (AddIn Onli, Azure AD application) can't be configured in the desired timeframes and no auth strategies work. A simple example might be 2FA (multi-factor authentication) or custom ADFS provider. As a quick workaround, the On-Demand auth can help.

On-Demand means that an interactive browser session is started where a user can provide the credentials as if he/she opens the SharePoint site and follows the same flow as reaching the site in a browser.

In that strategy, the application actually opens the browser and communicates via debug protocol for the auth cookies when uses them in the requests.

Lorca masters Chrome Debug Protocol, therefore, the Chrome/Chromium browser must be installed in the system where On-Demand auth is intended to be called.

Configure and usage sample

package main

import (
	"fmt"
	"log"
	"os"

	"github.com/koltyakov/gosip"
	"github.com/koltyakov/gosip/api"
	strategy "github.com/koltyakov/gosip-sandbox/strategies/ondemand"
)

func main() {

	authCnfg := &strategy.AuthCnfg{
		SiteURL: os.Getenv("SPAUTH_SITEURL"),
	}

	client := &gosip.SPClient{AuthCnfg: authCnfg}
	sp := api.NewSP(client)

	res, err := sp.Web().Select("Title").Get()
	if err != nil {
		log.Fatal(err)
	}

	fmt.Printf("Site title: %s\n", res.Data().Title)

}

On-Demand configuration assumes only SiteURL to be provided as everything else is dynamically resolved while the transition to the browser page.

The auth technique works for any strategy which is based on the cookies.

The strategy caches the cookies in the context of the SharePoint host. As a result, you won't see the credentials prompt each time. If it's not the desired behavior .CleanCookieCache() method can be called to clean the local cache.

Note, that the technique is only applicable when user interaction is assumed. Never ever use that auth approach in headless scenarios.

Struct

type AuthCnfg struct {
  // SPSite or SPWeb URL, which is the context target for the API calls
  SiteURL      string `json:"siteUrl"`
  Username     string `json:"username"`
  Password     string `json:"password"`
  // Following are not required for SPO
  Domain       string `json:"domain"`
  RelyingParty string `json:"relyingParty"`
  AdfsURL      string `json:"adfsUrl"`
  AdfsCookie   string `json:"adfsCookie"`
}

Gosip's ADFS also supports a scenario of ADFS or NTML behind WAP (Web Application Proxy) which adds additional auth flow and EdgeAccessCookie involved into play.

JSON

On-Premises configuration

private.json sample:

{
  "siteUrl": "https://www.contoso.com/sites/test",
  "username": "john.doe@contoso.com",
  "password": "this-is-not-a-real-password",
  "relyingParty": "urn:sharepoint:www",
  "adfsUrl": "https://login.contoso.com",
  "adfsCookie": "FedAuth"
}

On-Premises behing WAP configuration

private.json sample:

{
  "siteUrl": "https://www.contoso.com/sites/test",
  "username": "john.doe@contoso.com",
  "password": "this-is-not-a-real-password",
  "relyingParty": "urn:AppProxy:com",
  "adfsUrl": "https://login.contoso.com",
  "adfsCookie": "EdgeAccessCookie"
}

SharePoint Online configuration

private.json sample:

{
  "siteUrl": "https://contoso.sharepoint.com/sites/test",
  "username": "john.doe@contoso.onmicrosoft.com",
  "password": "this-is-not-a-real-password"
}

Code sample

package main

import (
	"log"
	// "os"

	"github.com/koltyakov/gosip"
	strategy "github.com/koltyakov/gosip/auth/adfs"
)

func main() {
	// authCnfg := &strategy.AuthCnfg{
	// 	SiteURL:  os.Getenv("SPAUTH_SITEURL"),
	// 	Username: os.Getenv("SPAUTH_USERNAME"),
	// 	Password: os.Getenv("SPAUTH_PASSWORD"),
	//	/* other auth props ...*/
	// }
	// or using `private.json` creds source

	authCnfg := &strategy.AuthCnfg{}
	configPath := "./config/private.json"
	if err := authCnfg.ReadConfig(configPath); err != nil {
		log.Fatalf("unable to get config: %v", err)
	}

	client := &gosip.SPClient{AuthCnfg: authCnfg}
	// use client in raw requests or bind it with Fluent API ...
}

It's not an auth strategy but a mode without any authentication flow applied to the SPClient.

Anonymous mode can be handy in a situation when Gosip SharePoint-aware helpers intended to be used however authentication is handled by any other middleware.

Struct

Only SiteURL is required.

JSON

private.json sample:

Code sample

Usage sample

package main

import (
	"fmt"
	"log"
	"os"

	"github.com/koltyakov/gosip"
	"github.com/koltyakov/gosip/api"
	strategy "github.com/koltyakov/gosip-sandbox/strategies/ntlm"
)

func main() {

	authCnfg := &strategy.AuthCnfg{
		SiteURL:  os.Getenv("SPAUTH_SITEURL"),
		Username: os.Getenv("SPAUTH_USERNAME"),
    Domain:   os.Getenv("SPAUTH_DOMAIN"),
		Password: os.Getenv("SPAUTH_PASSWORD"),
	}

	client := &gosip.SPClient{AuthCnfg: authCnfg}
	sp := api.NewSP(client)

	res, err := sp.Web().Select("Title").Get()
	if err != nil {
		log.Fatal(err)
	}

	fmt.Printf("Site title: %s\n", res.Data().Title)

}

Use the alternative only as a fallback method if go-ntlmssp doesn't work and don't forget to post an issue to help making Open Source tools better.

Provides a simple way of constructing API endpoint calls with IntelliSense and chainable syntax.

Usage sample

package main

import (
	"encoding/json"
	"fmt"
	"log"

	"github.com/koltyakov/gosip"
	"github.com/koltyakov/gosip/api"
	strategy "github.com/koltyakov/gosip/auth/addin"
)

func main() {
	// Getting auth params and client
	client, err := getAuthClient()
	if err != nil {
		log.Fatalln(err)
	}

	// Binding SharePoint API
	sp := api.NewSP(client)

	// Custom headers (optional)
	headers := map[string]string{
		"Accept": "application/json;odata=minimalmetadata",
		"Accept-Language": "de-DE,de;q=0.9",
	}
	config := &api.RequestConfig{Headers: headers}

	// Chainable request sample
	data, err := sp.Conf(config).Web().Lists().Select("Id,Title").Get()
	if err != nil {
		log.Fatalln(err)
	}

	// Response object unmarshalling
	// struct depends on OData mode and API method
	res := &struct {
		Value []struct {
			ID    string `json:"Id"`
			Title string `json:"Title"`
		} `json:"value"`
	}{}

	if err := json.Unmarshal(data, &res); err != nil {
		log.Fatalf("unable to parse the response: %v", err)
	}

	for _, list := range res.Value {
		fmt.Printf("%+v\n", list)
	}
}

func getAuthClient() (*gosip.SPClient, error) {
	configPath := "./config/private.spo-addin.json"
	auth := &strategy.AuthCnfg{}
	if err := auth.ReadConfig(configPath); err != nil {
		return nil, fmt.Errorf("unable to get config: %v", err)
	}
	return &gosip.SPClient{AuthCnfg: auth}, nil
}

Main concepts

• Get authenticated

• Construct root SP object using api.NewSP(client)

• Construct API calls in a fluent way

• Parse responses in the Go way

• Embrase strongly typed generic responses

• Build awesome apps in Go for SharePoint

Gosip HTTP client is preconfigured with retry policies so different kinds of failures can be back off. Current implementation of policies assumes a specific number of retries for a specific response HTTP Status Code. One would only consider retries for "non-ok" status codes and only those which represents server or networking error which has chances for backing off on a next try.

Defaults

The default policies are:

Retries' delay increases with each attempt: 200ms, 400ms, 800ms, 1.6s, 3.2s, and so on using the following progression formula:

For the responses with Retry-After header, retry after value is used in preference as threshold is usually require longer wait until next request permitted.

Custom policy

A custom policy can be provided on demand in the following way:

Disabling retries for a specific request

When a specific request assumes no retries, e.g. resolving a folder which potentially doesn't exist but you know that 500 is returned in case of failure and it's a faster algorithm to try optimistically and check or create on an error only, it can be handy disabling any retries but only scoped to a specific request or series of requests.

For that purposes X-Gosip-NoRetry header is the resolution.

Custom retry delays

While adding timeouts in hooks, make sure you're using something respecting context cancelation:

Gosip HTTP client is SharePoint nuances-aware, it takes care under the hood of such things as headers, API calls retries, threshholds, error handling, POST API requests Digests, and, of course, authentication and its renewal.

Usage

Methods

HTTP Client methods covers those which used in SharePoint all the way.

In a contract to default Go http.Client's .Do method, Gosip HTTP Client's methods proceed and read response body to return array of bytes. Which reduce amount of scaffolded code a bit and more handy for the REST API consumption, in our opinion, which is 90% of use cases.

Get

Sends GET request, embeds "Accept": "application/json;odata=verbose" header as a default. Use it for REST API GET calls.

Post

Sends POST request, embeds "Accept": "application/json;odata=verbose" and "Content-Type": "application/json;odata=verbose;charset=utf-8" headers as default. X-RequestDigest is received, cached, and embed automattically. Use it for REST API POST calls.

Delete

Sends POST request, embeds "Accept": "application/json;odata=verbose", "Content-Type": "application/json;odata=verbose;charset=utf-8", and "X-Http-Method": "DELETE" headers as default. X-RequestDigest is received, cached, and embed automattically. Use it for REST API POST calls with delete resource intention.

Update

Sends POST request, embeds "Accept": "application/json;odata=verbose", "Content-Type": "application/json;odata=verbose;charset=utf-8", and "X-Http-Method": "MERGE" headers as default. X-RequestDigest is received, cached, and embed automattically. Use it for REST API POST calls with update resource intention.

ProcessQuery

Sends POST request to /_vti_bin/client.svc/ProcessQuery endpoint (CSOM). All required headers for a CSOM request are embed automatically. Method's body should stand for a valid CSOM XML package. The response body is parsed for error handling, yet returned in it's original form.

Low-level HTTP client

Sometimes more control is required, e.g. when downloading files or large responses you could prefer precessing a response in chunks. This can be achieved with the low-level usage of Gosip HTTP client.

SPClient has Execute method which is a wrapper function injecting SharePoint authentication and ending up calling http.Client's Do method.

So you can dive down to native *http.Request at this point it's just a standard request from "net/http" package, but authenticated to SharePoint and with some batteries under the hood for a seemles API integration.

Low level client

client := &gosip.SPClient{AuthCnfg: auth}

var req *http.Request
// Initiate API request
// ...

req = req.WithContext(context.Background()) // <- pass a context

resp, err := client.Execute(req)
if err != nil {
  fmt.Printf("Unable to request api: %v", err)
  return
}

HTTP Client

While using HTTPClient, context is defined together with request config.

spClient := api.NewHTTPClient(&gosip.SPClient{AuthCnfg: auth})

endpoint := auth.GetSiteURL() + "/_api/web?$select=Title"

reqConf := &api.RequestConfig{
  Context: context.Background(), // <- pass a context
}

data, err := spClient.Get(endpoint, reqConf)
if err != nil {
  log.Fatalf("%v\n", err)
}

// spClient.Post(endpoint, body, reqConf) // generic POST

// generic DELETE helper crafts "X-Http-Method"="DELETE" header
// spClient.Delete(endpoint, reqConf)

// generic UPDATE helper crafts "X-Http-Method"="MERGE" header
// spClient.Update(endpoint, body, reqConf)

// CSOM helper (client.svc/ProcessQuery)
// spClient.ProcessQuery(endpoint, body, reqConf)

Fluent API

With Fluent API, context is managed in the same way as in the previous example with the only difference how it is chained to fluent syntax.

config := &api.RequestConfig{
  Context: context.Background(), // <- pass a context
}

sp := api.NewSP(client).Conf(config)

data, err := sp.Web().Lists().Select("Id,Title").Get()
if err != nil {
	log.Fatalln(err)
}

Conf method can be used almost on any hierarchy level. It's inherited with capability to redefine.

Binding authentication and API client

Understand SharePoint environment type and authentication strategy

If you have no idea which strategy is used within your farm the question is better be addressed to SharePoint admins.

Let's assume it's SharePoint Online and Add-In Only permissions. Then strategy "github.com/koltyakov/gosip/auth/addin" sub package should be used.

package main

import (
  "github.com/koltyakov/gosip"
  "github.com/koltyakov/gosip/api"
  strategy "github.com/koltyakov/gosip/auth/addin"
)

It could have been SharePoint On-Premise and NTLM and strategy "github.com/koltyakov/gosip/auth/ntlm" as imported strategy.

Initiate authentication object

Different authentication strategies assumes different credential parameters. Sometimes it can be Username/Password, sometimes CertPath or ClientID/ClientSecret. Please refer a specific strategy documentation for relevalt for the auth type parameters.

Credential can be passed directly in AuthCnfg's struct. Here a sample for addin:

auth := &strategy.AuthCnfg{
  SiteURL:      os.Getenv("SPAUTH_SITEURL"),
  ClientID:     os.Getenv("SPAUTH_CLIENTID"),
  ClientSecret: os.Getenv("SPAUTH_CLIENTSECRET"),
}

configPath := "./config/private.json"
auth := &strategy.AuthCnfg{}

err := auth.ReadConfig(configPath)
if err != nil {
  fmt.Printf("Unable to get config: %v\n", err)
  return
}

Bind auth client with Fluent API

Now when auth is bound it should be passed to client and Fluent API instance:

client := &gosip.SPClient{AuthCnfg: auth}

sp := api.NewSP(client)

Most of the samples starts with sp assuming configuration described above is already in place.

res, err := sp.Web().Select("Title").Get()
if err != nil {
  fmt.Println(err)
}

fmt.Printf("%s\n", res.Data().Title)

For large documents it's better using files Chunk API which allows splitting upload into separate REST API requests reducing memory consumption and providing a more manageable upload mechanism.

Files Chunk API appeared in SharePoint 2016 and should not be expected to work in the previous version.

Basic chunk upload example

file, err := os.Open("/path/to/large/file.zip")
if err != nil {
	log.Fatalf("unable to read a file: %v\n", err)
}
defer file.Close()

fileAddResp, err := foler.Files().AddChunked("My File.zip", file, nil)
if err != nil {
	log.Fatal(err)
}

fmt.Printf("New file URL: %s\n", fileAddResp.Data().ServerRelativeURL)

Upload flow

Fluent API abstracts some aspects of the chunked upload. Internally, /StartUpload, /ContinueUpload, /CancelUpload, and /FinishUpload endpoints are used. However, these are not exposed for the simplicity. Nonetheless, there is a way for canceling upload with the help of api.AddChunkedOptions.

Add chunked options

There are a few options which are optional. The third method parameter is a configuration structure.

type AddChunkedOptions struct {
	Overwrite bool // should overwrite existing file
	// on progress callback, execute custom logic on each chunk
	// if the Progress is used it should return "true" to continue upload
	// otherwise upload is canceled
	Progress  func(data *FileUploadProgressData) bool
	ChunkSize int // chunk size in bytes
}

Defaults are: Overwrite is true and ChunkSize equals to 10485760 bytes.

Progress callback allows not only trigger a progress logic, for example upload percentage update, but also to cancel an upload. To cancel an upload the progress callback should return false.

filePath := "/path/to/large/file.zip"

file, err := os.Open(filePath)
if err != nil {
	log.Fatalf("unable to read a file: %v\n", err)
}
defer file.Close()

info, _ := os.Stat(filePath)

options := &api.AddChunkedOptions{
	Overwrite: false,
	ChunkSize: 5 * 1024 * 1024,
	Progress: func(data *api.FileUploadProgressData) bool {
		fmt.Printf(
			"Block %d, sent %d%%\n",
			data.BlockNumber,
			100*data.FileOffset/fi.Size(),
		)
		return true // to cancel an upload on some external condition, return "false"
	},
}

if _, err := foler.Files().AddChunked("My File.zip", file, options); err != nil {
	log.Fatal(err)
}

SharePoint is ECM (Enterprise Content Management) system and it's common to expect files being uploaded, downloaded, migrated, processes, and managed in a variety ways.

Gosip provides an easy way of dealing with SharePoint document listaries, files and folders.

Getting library object

Document library in SharePoint is almost the same as a List, but with intention of being a container for files.

// The recommended way of getting lists is by using their relative URIs
// can be a short form without full web relative URL prefix
list := sp.Web().GetList("MyLibrary")

// other common but less recommended way of getting a list is
// list := sp.Web().Lists().GetByTitle("My Library")

Getting folder object

foler := sp.Web().GetFolder("MyLibrary/Folder01")

Getting file object

file := sp.Web().GetFile("MyLibrary/Folder01/File01.txt")

Adding new folder

// folderResp is a byte array read from response body with extra methods
folderResp, err := foler.Folders().Add("New Folder Name")
if err != nil {
	log.Fatal(err)
}

fmt.Printf("New folder URL: %s\n", folderResp.Data().ServerRelativeURL)

Deleting folders

folderRelativeURL := "MyLibrary/Folder01/New Folder Name"
if _, err := sp.Web().GetFolder(folderRelativeURL).Delete(); err != nil {
	log.Fatal(err)
}

Adding/uploading a file

// fileAddResp is a byte array read from response body with extra methods
fileAddResp, err := foler.Files().
	Add("My File.txt", []byte("File content"), true)

if err != nil {
	log.Fatal(err)
}

fmt.Printf("New file URL: %s\n", fileAddResp.Data().ServerRelativeURL)

Obviously, file content can be a result of reading a file from disk, e.g.:

content, err := ioutil.ReadFile("/path/to/file.txt")
if err != nil {
	log.Fatal(err)
}

fileAddResp, err := foler.Files().Add("My File.txt", content, true)
if err != nil {
	log.Fatal(err)
}

For the large files it's better using AddChunked API, hovewer, it was not available in SharePoint 2013.

file, err := os.Open("/path/to/large/file.zip")
if err != nil {
	log.Fatalf("unable to read a file: %v\n", err)
}
defer file.Close()

fileAddResp, err := foler.Files().AddChunked("My File.zip", file, nil)
if err != nil {
	log.Fatal(err)
}

fmt.Printf("New file URL: %s\n", fileAddResp.Data().ServerRelativeURL)

Downloading files

fileRelativeURL := "MyLibrary/Folder01/File01.txt"
data, err := sp.Web().GetFile(fileRelativeURL).Download()
if err != nil {
	log.Fatal(err)
}

file, err := os.Create("/path/toLocal/file.txt")
if err != nil {
	log.Fatalf("unable to create a file: %v\n", err)
}
defer file.Close()

_, err = file.Write(data)
if err != nil {
	log.Fatalf("unable to write to file: %v\n", err)
}

file.Sync()

For a large files it's better getting file reader through:

fileRelativeURL := "MyLibrary/Folder01/File01.txt"
fileReader, err := sp.Web().GetFile(fileRelativeURL).GetReader()
if err != nil {
	log.Fatal(err)
}
defer fileReader.Close()

file, err := os.Create("/path/toLocal/file.txt")
if err != nil {
	log.Fatalf("unable to create a file: %v\n", err)
}
defer file.Close()

if _, err := io.Copy(file, fileReader); err != nil {
	log.Fatalf("unable to save a file: %v\n", err)
}

Summary

Using Gosip you can concentrate on business logic and Go language aspects while processing documents actions in SharePoint seemlessly.

With use of IntelliSense and Fluent syntax other supported actions can be consumed based on a specific requirement.

Gosip allows providing custom authentication mechanisms. For example, you are considering reusing Fluent API helpers and HTTP Client but existing authentication strategies do not feet your environment specifics. Maybe your tenant configured with custom ADFS provider, maybe it's 2FA and there are no alternatives and you need On-Demand auth, but it missed in Gosip strategies list? Fortunately, this is not any sort of stopper. All included authentication strategies are a sort of a pluging and it's rather affordable to add a new strategy on your own.

Let's take a look at any strategy binding:

What we can see? Some strategy is imported into the strategy namespace. A strategy has AuthCnfg struct with some public properties which are obviously taking place in authentication flow. This struct is then passed to &gosip.SPClient{AuthCnfg: authCnfg} and somehow after following binding the requests are authenticated.

For this construction to work strategy.AuthCnfg should implement gosip.AuthCnfg interface which is:

Philosophy of the strategies is to have two initiation modes, the first is a strict declaration of the creds and the second one is reading credentials from the config. That config is not necessarily a file on the file system it can be a request to a key vault or OS credential manager, etc.

As the interface is passed to gosip.SPClient struct, Gosip knows nothing about the creds and the context, for that reason GetSiteURL method is vital to target requests to a correct root URL.

GetStrategy method should return the string alias value of the strategy name if something specific should be happening based on its value.

By implementing AuthCnfg struct and gosip.AuthCnfg interface any custom authentication can be added to Gosip.

Gosip provides an events system with a set of handlers that can be optionally defined to track different client communication aspects such as request tracking, retries and response error logging, to name just a few.

To define the handlers Hooks object should be configured and passed to gosip.SPClient struct.

var authCnfg gosip.AuthCnfg
// ... auth config initiation is omitted

&gosip.SPClient{
  AuthCnfg: authCnfg,
  Hooks:    &gosip.HookHandlers{
    // handlers function definition
  },
}

The following handlers are available at the moment:

// HookHandlers struct to configure events handlers
type HookHandlers struct {
	OnError    func(event *HookEvent) // when error appeared
	OnRetry    func(event *HookEvent) // before retry request
	OnRequest  func(event *HookEvent) // before request is sent
	OnResponse func(event *HookEvent) // after response is received
}

All of the handlers are optional.

A handler receives HookEvent pointer which contains request pointer, response status code, and error (if applicable for an event), and time information to track duration since a request started an event happened.

Hooks sample:

// Define requests hook handlers
client.Hooks = &gosip.HookHandlers{
	OnError: func(e *gosip.HookEvent) {
		fmt.Println("\n======= On Error ========")
		fmt.Printf(" URL: %s\n", e.Request.URL)
		fmt.Printf(" StatusCode: %d\n", e.StatusCode)
		fmt.Printf(" Error: %s\n", e.Error)
		fmt.Printf("  took %f seconds\n",
		  time.Since(e.StartedAt).Seconds())
		fmt.Printf("=========================\n\n")
	},
	OnRetry: func(e *gosip.HookEvent) {
		fmt.Println("\n======= On Retry ========")
		fmt.Printf(" URL: %s\n", e.Request.URL)
		fmt.Printf(" StatusCode: %d\n", e.StatusCode)
		fmt.Printf(" Error: %s\n", e.Error)
		fmt.Printf("  took %f seconds\n",
		  time.Since(e.StartedAt).Seconds())
		fmt.Printf("=========================\n\n")
	},
	OnRequest: func(e *gosip.HookEvent) {
		if e.Error == nil {
			fmt.Println("\n====== On Request =======")
			fmt.Printf(" URL: %s\n", e.Request.URL)
			fmt.Printf("  auth injection took %f seconds\n",
			  time.Since(e.StartedAt).Seconds())
			fmt.Printf("=========================\n\n")
		}
	},
	OnResponse: func(e *gosip.HookEvent) {
		if e.Error == nil {
			fmt.Println("\n====== On Response =======")
			fmt.Printf(" URL: %s\n", e.Request.URL)
			fmt.Printf(" StatusCode: %d\n", e.StatusCode)
			fmt.Printf("  took %f seconds\n",
			  time.Since(e.StartedAt).Seconds())
			fmt.Printf("==========================\n\n")
		}
	},
}

Hooks can be handy for global logging streaming and metrics collection.

It is recommended using asynchronous and only lightweight logic inside hooks.

CRUD is the most commonly requested type of API consumption.

This example demonstrate basic operations on a list items sample. Hovewer, SharePoint has a variety of nuances even when it comes to just getting items from a list.

Getting list object

// The recommended way of getting lists is by using their relative URIs
// can be a short form without full web relative URL prefix
list := sp.Web().GetList("Lists/MyList")

// other common but less recommended way of getting a list is
// list := sp.Web().Lists().GetByTitle("My List")

Getting items

// itemsResp is a byte array read from response body
// powered with some processing "batteries" as Data() or Unmarshal() methods
itemsResp, err := list.Items().
	Select("Id,Title").  // OData $select modifier, limit what props are retrieved
	OrderBy("Id", true). // OData $orderby modifier, defines sort order
	Top(10).             // OData $top modifier, limits page size
	Get()                // Finalizes API constructor and sends a response

if err != nil {
	log.Fatal(err)
}

// Data() method is a helper which unmarshals generic structure
// use custom structs and unmarshal for custom fields
for _, item := range itemsResp.Data() {
	itemData := item.Data()
	fmt.Printf("ID: %d, Title: %s\n", itemData.ID, itemData.Title)
}

Based on a use case, items can be reveived with alternative methods, as .GetAll, .GetPaged, .GetByCAML, or even lists methods as .RenderListData.

Adding an item

// Payload should be a valid JSON stringified string converted to byte array
// Payload's properties must be a valid OData entity types
itemPayload := []byte(`{
	"Title": "New Item"
}`)

// Constructs and sends add operation request
// itemAddRes is a byte array read from response body with extra methods
itemAddRes, err := list.Items().Add(itemPayload)
if err != nil {
	log.Fatal(err)
}

fmt.Printf("Raw response: %s\n", itemAddRes)
fmt.Printf("Added item's ID: %d\n", itemAddRes.Data().ID)

Payloads can be constructed in a usual Go way using marshalling struct or string maps to JSON. E.g.:

itemMetadata := &struct{
	Title string `json:"Title"`
}{
	Title: "New Item",
}

itemPayload, _ := json.Marshal(itemMetadata)

or:

itemMetadata := map[string]interface{}{
	"Title": "New Item",
}

itemPayload, _ := json.Marshal(itemMetadata)

up to your preferences.

Getting a specific item

itemID := 42 // should specific item ID

itemRes, err := list.Items().GetByID(itemID).Get()
if err != nil {
	log.Fatal(err)
}

fmt.Printf("Raw response: %s\n", itemRes)
fmt.Printf("Item metadata: %+v\n", itemRes.Data())

Updating an item

// Payload should be a valid JSON stringified string converted to byte array
// Payload's properties must be a valid OData entity types
itemUpdatePayload := []byte(`{
	"Title": "Updated Title"
}`)

itemID := 42 // should specific item ID

// Constructs and sends update operation request
// itemUpdateRes is a byte array read from response body with extra methods
itemUpdateRes, err := list.Items().GetById(itemID).Update(itemUpdatePayload)
if err != nil {
	log.Fatal(err)
}

fmt.Printf("Item is updated, %s\n", itemUpdateRes.Data().Modified)

Payloads can be constructed in a usual Go way using marshalling struct or string maps to JSON, same as in add operations.

Delete an item

Item can be not only deleted but recycled with a further restore operation, which can be provide more safety.

itemID := 42 // should specific item ID

// list.Items().GetByID(itemID).Delete() // or .Recycle()
if _, err := list.Items().GetByID(itemID).Recycle(); err != nil {
	log.Fatal(err)
}

Getting profiles

The most in-demand, I would say, feature when it comes to user profiles is getting all profiles. However, UPS API allows only dealing with a single profile, you can't request all of them.

Luckily, this is possible and recommended achieving with the search.

res, err := sp.Search().PostQuery(&api.SearchQuery{
	QueryText: "*",
	SourceID:  "b09a7990-05ea-4af9-81ef-edfab16c4e31",
})

if err != nil {
	log.Fatal(err)
}

fmt.Printf("%+v\n", res.Results())

Getting profile properties

user, err := sp.Web().SiteUsers().
	GetByEmail("jane.doe@contoso.onmicrosoft.com").Get()

if err != nil {
	log.Fatal(err)
}

props, err := sp.Profiles().GetPropertiesFor(user.Data().LoginName)
if err != nil {
	log.Fatal(err)
}

fmt.Printf("%+v\n", props.Data())

Gosip strongly types properties response using .Data() helper, here is the resulted struct:

type ProfilePropsInto struct {
	AccountName           string
	DirectReports         []string
	DisplayName           string
	Email                 string
	ExtendedManagers      []string
	ExtendedReports       []string
	Peers                 []string
	IsFollowed            bool
	PersonalSiteHostURL   string
	PersonalURL           string
	PictureURL            string
	Title                 string
	UserURL               string
	UserProfileProperties []*TypedKeyValue
}

Getting single property

Sometimes you have to be as effective as possible and trim down responses to a minimum. Let's say you only need a single property.

rop, err := sp.Profiles().
	GetUserProfilePropertyFor(user.Data().LoginName, "AccountName")

if err != nil {
	log.Fatal(err)
}

fmt.Printf("%s\n", prop)

Seriously, I don't know the reason for existing of getting a single property, but not getting specific properties or multiple profiles or updating multiple properties. There are no excuses yet this part of SharePoint API is clunky and really old.

Setting user profile property values

There are two methods for setting user profile property value. Yeah, you heard me the right property in a time.

Set single value profile property

if err := sp.Profiles().SetSingleValueProfileProperty(
	user.Data().LoginName,
	"AboutMe",
	"Updated from Gosip",
); err != nil {
	log.Fatal(err)
}

Set multi valued profile property

tags := []string{"#ci", "#demo", "#test"}
if err := sp.Profiles().SetMultiValuedProfileProperty(
	user.Data().LoginName,
	"SPS-HashTags",
	tags,
); err != nil {
	log.Fatal(err)
}

Summary

User profiles API is limited due to its legacy nature. It is what it is. However, many SharePoint solutions, especially intranet portals and workflow processes can be heavily based on UPS. Go worker can be handy for custom synchronization scenarios and also in external workflow workers with UPS as a source for settings for detecting user dynamic roles.

Our common recommendation with attachments is to use them in moderation with a preference to documents in libraries and linking business objects items to that document using metadata and other logical relationship. But sometimes you need nothing more than just a simple binary addition to an item.

Working with attachments is mostly straightforward as you can only get a list of item's attachment, get a specific attachment by its name, add and delete an attachment.

Getting attachments

list := sp.Web().GetList("Lists/MyList")
item := list.Items().GetByID(1)

attachments, err := item.Attachments().Get()
if err != nil {
	log.Fatal(err)
}

for _, attachment := range attachments.Data() {
	data := attachment.Data()
	fmt.Printf("%s (%s)\n", data.FileName, data.ServerRelativeURL)
}

Attachments API provides little information, actually only FileName and ServerRelativeURL.

Items have attachments

To detect which items have attachments the corresponding Attachments property can be requested within an ordinary get items request:

items, err := list.Items().Select("Id,Attachments").Get()
if err != nil {
	log.Fatal(err)
}

for _, item := range items.Data() {
	data := item.Data()
	hasAttachments := "has no"
	if data.Attachments {
		hasAttachments = "has"
	}
	fmt.Printf("Item ID %d %s attachments\n", data.ID, hasAttachments)
}

Adding attachments

list := sp.Web().GetList("Lists/MyList")
item := list.Items().GetByID(1)

content := strings.NewReader("Get content in a usual Go way you like")

item.Attachments().GetByName("MyAttachment.txt").Delete()

resp, err := item.Attachments().Add("MyAttachment.txt", content)
if err != nil {
	log.Fatal(err)
}

fmt.Printf("Attachment is added: %s\n", resp.Data().ServerRelativeURL)

Getting attachment by name

list := sp.Web().GetList("Lists/MyList")
item := list.Items().GetByID(1)

content, err := item.Attachments().GetByName("MyAttachment.txt").Download()
if err != nil {
	log.Fatal(err)
}

fmt.Printf(
	"Do whatever needed with this content of %d bytes\n",
	len(content),
)

Attachment actions

With an attachment you can:

• Download

• Get reader (download in a stream way)

• Delete

• or Recycle

These actions are rather obvious with the help of the Fluent API.

Permissions management is an important part of business logic which can be met in worker processes and workflows.

Let's explore how Gosip Fluent API wraps up SharePoint securable object and permissions.

First of all, by securable objects mean any SharePoint artifacts which can be configured with unique permissions: webs, lists, libraries, items, etc.

Fluent API scopes role operations under .Roles() method.

Checking is an object has unique permissions

Breaking role inheritance

To assign unique permissions to an object its role inheritence must be broken.

Role definitions

Before assigning permissions it's important to know how to get role definitions.

A role definition is a collection of rights bound to a specific object. Role definitions (for example, Full Control, Read, Contribute, Design, or Limited Access) are scoped to the Web site and mean the same thing everywhere within the Web site, but their meanings can differ between sites within the same site collection. Role definitions can also be inherited from the parent Web site, just as permissions can be inherited.

Definitions are csoped as Web level in .RoleDefinitions() quariable collection.

Definitions can be gotten by Definition ID, Name or Type. We recommend using OOTB definitions and getting them by types. There is an enumerator-like helper for default types api.RoleTypeKinds.

Getting principals

Principal is site user or group, their IDs are used in roles assignments. Principal ID can be received by requesting UIL (User Information List), getting site user/group by name/email, etc.

It's in preference to operate on groups level and grant permissions to a specific users as little as possible.

Adding role assignments

The role assignment is the relationship among the role definition, the users and groups, and the scope (for example, one user may be a reader on list 1, while another user is a reader on list 2). The relationship expressed through the role assignment is the key to making SharePoint security management role-based.

At last, now we are ready for roles assigment. 😜 Who told permissions is simple?

Removing role assignments

Removing role assignments is just the same as adding but in opposite.

Reseting roles inheritance

To reset permissions inheritance:

After reseting object roles assigments its permissions are again inherited from the parent object.

Getting collection objects' permissions

When dealing with OData collection of objects with unique permissions OData items' role assigment can be requested using RoleAssignments, also HasUniqueRoleAssignments can be used in moderation.

Please be aware that HasUniqueRoleAssignments is a heavy property that creates workload on a SharePoint server and better be used at minimal.

Try not abusing it by potential requests to large lists getting a bunch of items.

RoleAssigments if any applied contains an array of objects.

Assigments is a Member and binded RoleDefinitions:

Base permissions

BasePermissions is permissions representation with Low and High pair. Don't panic if API returns only BasePermissions ({ "High": "2147483647", "Low": "4294705151" }), using HasPermissions helper it's simple to check if it includes required permissions kind:

Summary

Now we know how to treat SharePoint permissions using Gosip and Fluent API. Before wrapping up, we want to stress on importance of careful planning of permissions model, as less unique permissions is better.

When it comes to synchronization or delta processing Change API is for the rescue. There is no need in any sort of logic that used modified date comparison to detect what has been changed since the last processing time. There is no need to wait until the search crawl is done to check changes through web or site collection. And not only these benefits but also tracking permissions changes restores from recycle bin, system updates and much more.

Changes API is what is used together with Webhooks (in SPO), by having a context (scope) and change token(s) you can easily retrieve what has been changed and run the custom logic based on the nature of changes.

Changes can be requested from within different scopes: lists, webs, sites, etc. Change API operates change tokens to establish start and end anchors.

Change tokens received from a specific entity type (e.g. Site) can't be used with another entity type (e.g. List) while sending change query.

Getting current change token

siteChangeToken, err := sp.Site().Changes().GetCurrentToken()
if err != nil {
  log.Fatal(err)
}

fmt.Printf("Site change token: %s\n", siteChangeToken)

webChangeToken, err := sp.Web().Changes().GetCurrentToken()
if err != nil {
  log.Fatal(err)
}

fmt.Printf("Web change token: %s\n", webChangeToken)

list := sp.Web().GetList("Lists/MyList")
listChangeToken, err := list.Changes().GetCurrentToken()
if err != nil {
  log.Fatal(err)
}

fmt.Printf("List change token: %s\n", listChangeToken)

// Site change token:
//   1;1;afbc6f5a-65b3-4b64-9f40-885d8d772c8c;637138201430100000;64696075
// Web change token:
//   1;2;7e084f68-c401-4910-b75e-4347c7848965;637138201430100000;64696075
// List change token:
//   1;3;3b542696-5863-4ff7-bb90-8224e9e8adb9;637138201430100000;64696075

Getting changes

Knowing what root entities' token(s) you have the specific changes query can be crafted and sent, e.g.:

list := sp.Web().GetList("Lists/MyList")
listChangeToken, _ := list.Changes().GetCurrentToken()
list.Items().Add([]byte(`{"Title":"New item"}`))
// error handling is omitted -- adding a dummy item to receive changes result

changes, err := list.Changes().GetChanges(&api.ChangeQuery{
  ChangeTokenStart: listChangeToken,
  List:             true,
  Item:             true,
  Add:              true,
})
if err != nil {
  log.Fatal(err)
}

for _, change := range changes.Data() {
  fmt.Printf("%+v\n", change)
}

Change results are strongly typed with Gosip's API, the changes variable in the sample is an array of api.ChangeInfo struct pointers.

Change into struct contains the following properties:

type ChangeInfo struct {
  ChangeToken       *StringValue
  ChangeType        int
  Editor            string
  EditorEmailHint   string
  ItemID            int
  ListID            string
  ServerRelativeURL string
  SharedByUser      string
  SharedWithUsers   string
  SiteID            string
  Time              time.Time
  UniqueID          string
  WebID             string
}

In a synchronization scenario or a Webhook after getting information which items are changed the corresponding request(s) should be sent for getting specifics.

Change query

When getting the changes the API requires some clarifications about what exactly should be retrieved. This is defined within a change query which is implemented api.ChangeQuery struct.

Pagination

Sometimes it can be lots of changes since a provided token. Gosip implements pagination helper using "jumping" between different change tokens under the hood. When last change item's token is used as a start token to get the "next page". This approach is used in GetNextPage:

changesFirstPage, _ := list.Changes().Top(100).GetChanges(&ChangeQuery{
	ChangeTokenStart: token,
	List:             true,
	Item:             true,
	Add:              true,
})

changesSecondPage, _ := changesFirstPage.GetNextPage()

Summary

Change API is a powerful mechanism and a robust way for processing delta changes that can and should be used in advanced and optimised synchronizations and business processes withing external workers.

Search and search API is a huge and complex topic. In this article we're going mostly cover some basics in combination of Gosip plus REST API.

Basic search call

While executing search call there is no difference what web/site is used as a context, search API returns data due to search query object.

The simplest request looks this way:

// `res`ult here is a byte array with some helper methods as .Results()
res, err := sp.Search().PostQuery(&api.SearchQuery{
    QueryText: "*",
    RowLimit:  5,
})

if err != nil {
    log.Fatal(err)
}

fmt.Printf("%+v\n", res.Results())

Search query

Search query struct (api.SearchQuery) contains some, let's say, reasonable amount of options 🙈.

But don't be afraid you use only a few of them and on demand and when know what you need.

User profiles search sample

res, err := sp.Search().PostQuery(&api.SearchQuery{
	QueryText:        "*",
	RowLimit:         5,
	SelectProperties: []string{"AccountName", "Title", "Department", "JobTitle"},
	StartRow:         10,
	TrimDuplicates:   false,
	SourceID:         "b09a7990-05ea-4af9-81ef-edfab16c4e31",
	SortList: []*api.SearchSort{
		&api.SearchSort{
			Property:  "Title",
			Direction: 0,
		},
	},
})

if err != nil {
	log.Fatal(err)
}

fmt.Printf("AccountName | Title | Department | JobTitle\n")
fmt.Printf("------------|-------|------------|---------\n")
for _, profile := range res.Results() {
	fmt.Printf(
		"%s | %s | %s | %s\n",
		profile["AccountName"],
		profile["Title"],
		profile["Department"],
		profile["JobTitle"],
	)
}

This search request searched only for user profiles (defined by SourceID), return five results per page, skips first ten records ignores trimming duplicates, retrieves specific managed properties and sorts by Title property.

Search response

Search response in Fluent API is sligtly bit adjusted with helper methods. Result itself is byte array, so you can process it in a custom way.

For lazy people (as I am) there are .Data() and .Results() helpers.

Results helper

Results helper retrieves PrimaryQueryResult.RelevantResults.Table.Rows and reduces search response results to the convenient array of string maps. So you can grasp results this way:

for _, profile := range res.Results() {
	fmt.Printf(
		"%s | %s | %s | %s\n",
		profile["AccountName"],
		profile["Title"],
		profile["Department"],
		profile["JobTitle"],
	)
}

Data helper

Data helper provides more options. It returns the following struct:

type SearchResults struct {
	ElapsedTime           int
	PrimaryQueryResult    *ResultTableCollection
	Properties            []*TypedKeyValue
	SecondaryQueryResults []*ResultTableCollection
	SpellingSuggestion    string
	TriggeredRules        []interface{}
}

type ResultTableCollection struct {
	QueryErrors        map[string]interface{}
	QueryID            string
	QueryRuleID        string
	CustomResults      *ResultTable
	RefinementResults  *ResultTable
	RelevantResults    *ResultTable
	SpecialTermResults *ResultTable
}

So you can have access to refiners and many more search goodies.

In enterprise content management (ECM) scenarios, records are an important element within a document lifecycle and a company's compliance policies. Not only documents but actually items.

When declared as a record a document or item can't be later changed or moved without undeclaring. This guarantees integrity.

REST API in terms of managing classic in-place records is limited, you can only retrieve record status from item metadata. Item's OData__vti_ItemDeclaredRecord property tells was an item declared as a record and when. The field is a date-time value, empty means item is not a record.

We would not be we if didn't add an extension with allows Gosip to declare, undeclare and declare with a declaration date. To workaround REST's limitation, these functionality is achieved by corresponding CSOM calls. What's great is that Gosip abstracts for you such a nuance providing methods which under-the-hood details are interesting but not necessarily should be known by a library consumer.

Is item declared as record

list := sp.Web().GetList("Lists/MyList")
item := list.Items().GetByID(1)

isRecord, err := item.Records().IsRecord()
if err != nil {
	log.Fatal(err)
}

fmt.Printf("Item is record? %t\n", isRecord)

Is document declared as a record

Declaration as records is more common for documents (files), yet it's actually a file's item that keeps record status. And therefore holds API actions.

file := sp.Web().GetFile("MyLibrary/Contract.docx")
item, err := file.GetItem()
if err != nil {
	log.Fatal(err)
}

isRecord, err := item.Records().IsRecord()
if err != nil {
	log.Fatal(err)
}

fmt.Printf("Document is record? %t\n", isRecord)

Declare as a record

	if err := item.Records().Declare(); err != nil {
		log.Fatal(err)
	}

Undeclare as a record

if err := item.Records().Undeclare(); err != nil {
	log.Fatal(err)
}

Declare with declaration date

date, _ := time.Parse(time.RFC3339, "2019-01-01T08:00:00.000Z")
if err := item.Records().DeclareWithDate(date); err != nil {
	log.Fatal(err)
}

Summary

Property bags are a nice way of storing global metadata and settings in SharePoint. Property bags are key-value pairs scoped to the container. In general, any folder can act as a container, also webs have their own property bag storage located in all properties section.

Use-cases can vary depending on how an application uses this key-value storage. For example, PnP Provisioning engine stores applied schema information in property bags. A good thing is that you don't need to provision any additional artifacts to keep some business logic or state variables.

A thing to know that all the props values are strings, dealing with different data types they should be serialized or converted to a string.

Getting web's all properties

props, err := sp.Web().AllProps().Get()
if err != nil {
	log.Fatal(err)
}

for key, val := range props.Data() {
	fmt.Printf("%s: %s\n", key, val)
}

Getting specific properties

To get a limited subset of properties .GetProps helper method can be used:

props, err := sp.Web().AllProps().GetProps([]string{
	"taxonomyhiddenlist",
	"vti_defaultlanguage",
})

if err != nil {
	log.Fatal(err)
}

for key, val := range props {
	fmt.Printf("%s: %s\n", key, val)
}

Selecting specific properties didn't work in old versions of SharePoint, however, the method has a fallback to getting all and filtering props on the client.

Setting property value

if err := sp.Web().AllProps().Set("my-prop", "my value"); err != nil {
	log.Fatal(err)
}

Modern SharePoint sites by default have custom scripting disabled mode. When custom scripting is disabled even an admin account will receive "Access denied. You do not have permission to perform this action or access this resource." error message. This is the expected behavior.

spo site classic set --url https://contoso/sites/site --noScriptSite false

Getting folder property bags

props, err := sp.Web().GetFolder("MyLibrary").Props().Get()
if err != nil {
	log.Fatal(err)
}

for key, val := range props.Data() {
	fmt.Printf("%s: %s\n", key, val)
}

Setting many properties values

err := sp.Web().GetFolder("MyLibrary").Props().
	SetProps(map[string]string{
		"prop01": "value 01",
		"prop02": "value 02",
	})

if err != nil {
	log.Fatal(err)
}

Summary

Property bags are a robust way of storing custom settings and state which requires no additional artifacts. When structuring and consuming correctly they can be a great addition to the application logic.

Site groups and users can be requested via Web's .SiteGroups() and .SiteUsers() queriable collections correspondingly.

Getting users sample

Getting groups sample

Ensuring a user

You can't add new user via SharePoint API, but a user who exists in AD/AAD can be added to a site by ensuring him/her by a logon name.

Associated groups

We love the "power of defaults". Each web by default has three predefined groups: Owners, Members and Visitors. But their IDs and names are different from web to web. Luckily there is a helper for getting associated groups.

Creating groups

Deleting groups

Adding user to a group

In group's .AddUser method the argument should be full and valid login name including security provider membership prefix. For instance, while you can ensure Jane using jane.doe@contoso.onmicrosoft.com the same as an .AddUser method will fail as Jane's login is actually different i:0#.f|membership|jane.doe@contoso.onmicrosoft.com.

If you already know UserID but not sure about LoginName .AddUserByID helper is at the disposal.

Removing users from a group

Similarly as with adding users:

Managing group owner

Getting group's users

Summary

That's it, most of the common actions with groups and users are covered.

You'd probably will be interested with the connected topics: