1

I want to manupulate (read, write, delete) files in a document library of my sharepoint (hosted at Sharepoint Online). Ultimately this will be Part of a Java Webapplication where a backend job runs regulary, generates Files and uploads them to Sharepoint. For now I'm working in a JUnit Test to get familiar with Sharepoints REST API.

And this is where trouble comes in.

What I tried so far…

1. SPOnline Library

I found this little library and tried it

// This connects to https://login.microsoftonline.com/extSTS.srf
// Token contains Cookies rtFa and FedAuth that will be added to all future requests
// Username and password are those credentials I would use to login at mysharepoint.sharepoint.com
String domain = "mysharepoint";
Pair<String, String> token = SPOnline.login(USERNAME, PASSWORD, domain);

// Connects to mysharepoint.sharepoint.com/_api/contextinfo
// Returns a form digest value that looks okay: 0xBIGLONGHEXADECIMALSTRING,27 May 2020 08:58:09 -0000
String jsonString = SPOnline.post(token, domain, "_api/contextinfo", null, null);
JSONObject json = new JSONObject(jsonString);
String formDigestValue = json.getJSONObject("d").getJSONObject("GetContextWebInformation").getString("FormDigestValue");

String path = "/sites/MySite/_api/web/GetFolderByServerRelativeUrl('/MySite/Test-Doc-Lib/Test-Folder/Test-File.xlsx')";

String deleteResponseJson = SPOnline.delete(token, domain, path, formDigestValue);

The Delete Request returns a 403 error. The response JSON says

The security validation for this page is invalid. Click Back in your Web browser, refresh the page, and try your operation again.

2. OAuth and Bearer Token, App Registration in Sharepoint

(To be honest I'm not sure if SPOnline doesn't use OAuth as well but at least there is no Bearer Token involved there)

According to the second link I do the following:

  • I go to https://mysharepoint.sharepoint.com/sites/MySite/_layouts/15/appregnew.aspx generate Client-ID and Client Secret and set Title=Test-SP-App, App-Domain=localhost, Redirect-URI=https://localhost.
  • Next I go to https://softwareschmiederaeder.sharepoint.com/sites/MySite/_layouts/15/appinv.aspx and grant permissions.
  • With a call to https://mysharepoint/sharepoint.com/_vti_bin/client.svc/ I get the tenant ID
  • With a call to https://accounts.accesscontrol.windows.net/<TenantID>/tokens/OAuth/2 I get a Bearer Access token

When Accessing any Sharepoint Ressource with this token I get a 403 error

{
    "error": {
        "code": "-2147024891, System.UnauthorizedAccessException",
        "message": {
            "lang": "de-DE",
            // Translation: Access denied. You don't have permission to execute this process or access this ressource.
            "value": "Zugriff verweigert. Sie haben keine Berechtigung, diesen Vorgang auszuführen oder auf diese Ressource zuzugreifen."
        }
    }
}

3. App registration in Azure

This post is about registering... something in Azure and then granting permissions to Sharepoint. I get to the point where I get a new access token in postman. A windows opens and asks for my credentials (which is not even acceptable, as I need to run this in a Java backend job where no user is present). When I enter my cretendials the login fails.

4. JShare

I also tried JShare which is not free but has 30 day trial, hoping to do a proof of concept.

Service service = new Service("https://mysharepoint.sharepoint.com", USERNAME, PASSWORD);
Folder folder = service.createFolder("/TestDocLib/CreatedFolder");

Again I get a 403 error.

5. JCIFS

I followed this post... again 403 error.

Solution Ideas

I'm a littly overwhelmed with these many possibilities. Right now I could imagine the following:

  • The account I use (USERNAME, PASSWORD) to login with Sharepoint dosn't have correct permissions. Since I'm the owner of the site MySite I somehow doubt it. I even tried giving me explicit permission by right click on Test-Folder and Manage Access.
  • I send my requests to the wrong urls. In the tutorial with my first attempt (SPOnline) the urls only contain /mysite/... while browsing my sharepoint always shows the prefix "site": /site/mysite. However if I remove that prefix I get a 404 error so I there is not even something to authenticate to.
Improve this question
2
  • I realize I'm supposed to provide an "answer", but I've only got a question to help me understand the answer... How and where is the code used? It appears very elegant and comprehensive, but I'm new enough to Java that I don't know where it belongs. I've tried it in JSFiddle, but of course get many errors. If you point me in the right direction, I can usually tough it out. Thanks, Scott
    – Scott Johnson
    Commented Aug 25, 2020 at 19:26
  • AuthHandler is a class on its own. Other classes may use it like: String bearerToken = authHandler.authenticate(); - Then you have the bearer token wich you can use to create a Request (I used HttpGet, HttpPost etc from Apache Commons Http). If your question is how to create a request, that would be far beyond the scope of this question, but feel free to create one and link it here.
    – Ruik
    Commented Sep 6, 2020 at 11:20

1 Answer 1

Reset to default
1

Finally figured this out. My solution depends on my attempt No. 2 and this tutorial:

http://www.ktskumar.com/2017/01/access-sharepoint-online-using-postman/

As I wrote, I already got the bearer token. I just couldn't use it because of the 403 error. In the last section Access the SharePoint resource instead of this URL

https://<sitename>.sharepoint.com/_api/web?$select=Title

I use this URL

https://<sitename>.sharepoint.com/sites/MySite/_api/web?$select=Title

Since I access the REST API as a subpath to my Sharepoint site, it works.

Code

This is my Code to retrieve the bearer token:

public class AuthHandler {

    private final Config config;

    public String authenticate() {
        CloseableHttpClient httpClient = HttpClients.createDefault();

        Pair<String, String> bearerRealmAndRessourceId = getBearerRealmAndRessourceId(httpClient);
        String bearerRealm = bearerRealmAndRessourceId.getLeft();
        String ressourceId = bearerRealmAndRessourceId.getRight();

        String bearerToken = getBearerToken(bearerRealm, ressourceId, httpClient);
        return bearerToken;
    }

    private String getBearerToken(String bearerRealm, String ressourceId, CloseableHttpClient httpClient) {
        String url = String.format("https://accounts.accesscontrol.windows.net/%s/tokens/OAuth/2", bearerRealm);

        HttpPost postRequest = new HttpPost(url);
        postRequest.setHeader("Content-Type", "application/x-www-form-urlencoded");

        String clientId = String.format("%s@%s", config.getClientId(), bearerRealm);
        String resource = String.format("%s/%s@%s", ressourceId, config.getDomain(), bearerRealm);
        List<NameValuePair> params = Lists.newArrayList(
            new BasicNameValuePair("grant_type", "client_credentials"),
            new BasicNameValuePair("client_id", clientId),
            new BasicNameValuePair("client_secret", config.getClientSecret()),
            new BasicNameValuePair("resource", resource)
        );

        try {
            postRequest.setEntity(new UrlEncodedFormEntity(params));
        } catch (UnsupportedEncodingException e) {
            throw new RuntimeException("Parameter falsch formatiert", e);
        }

        try  {
            HttpResponse response = httpClient.execute(postRequest);

            String bodyJson = IOUtils.toString(response.getEntity().getContent(), Charset.defaultCharset());
            JSONObject body = new JSONObject(bodyJson);
            String bearerToken = body.getString("access_token");
            return bearerToken;
        } catch (IOException e) {
            throw new RuntimeException("Post Request zum Holen des Bearer Tokens fehlgeschlagen", e);
        }
    }

    private Pair<String, String> getBearerRealmAndRessourceId(CloseableHttpClient httpClient) {
        // domain = mysharepoint.sharepoint.com
        String url = String.format("https://%s/_layouts/15/sharepoint.aspx", config.getDomain());

        HttpGet getRequest = new HttpGet(url);
        getRequest.setHeader("Authorization", "Bearer");

        try {
            HttpResponse response = httpClient.execute(getRequest);
            Header[] headers = response.getHeaders("www-authenticate");

            String bearerRealm = extractHeaderElement(headers, "Bearer realm");
            String ressourceId = extractHeaderElement(headers, "client_id");
            return Pair.of(bearerRealm, ressourceId);
        } catch (IOException e) {
            throw new RuntimeException("Get Request zum Holen von Bearer realm und client_id fehlgeschlagen", e);
        }
    }

    private String extractHeaderElement(Header[] headers, String elementName) {
        return Arrays.asList(headers).stream()
                .map(header -> header.getElements())
                .flatMap(elements -> Arrays.asList(elements).stream())
                .filter(element -> element.getName().equals(elementName))
                .findFirst()
                .orElseThrow()
                .getValue();
    }
}

And the relevant Maven dependencies:

    <dependency>
        <groupId>org.apache.httpcomponents</groupId>
        <artifactId>httpclient</artifactId>
        <version>4.5.12</version>
    </dependency>
    <dependency>
        <groupId>commons-io</groupId>
        <artifactId>commons-io</artifactId>
        <version>2.6</version>
    </dependency>
    <dependency>
        <groupId>org.json</groupId>
        <artifactId>json</artifactId>
        <version>20200518</version>
    </dependency>
Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.