Automated cost accounting batch import via scripting fails with token expiration

ERP Microsoft Dynamics 365
, , , , , , , ,
1

We’ve implemented an automated batch import script for cost accounting entries that runs via Windows Task Scheduler every night at 2 AM. The script authenticates using OAuth2 and pulls data from our ERP to process cost allocations across multiple departments.

The issue: Manual execution during business hours works perfectly, but scheduled overnight runs consistently fail with authentication errors. Looking at logs, the OAuth2 token appears to expire after 60 minutes, but our batch process takes 75-90 minutes to complete for large datasets.

When I run the same script manually during the day, it completes successfully even with the same data volume. The scheduled task runs under a service account with proper API permissions. Error log shows:


HTTP 401: Unauthorized
OAuth token validation failed
Timestamp: 03:17:42 (77 minutes after start)

We’re on D365 10.0.41. Has anyone dealt with token refresh in long-running unattended scripts? The script runs unattended so there’s no user interaction possible for re-authentication.

system 2

Here’s a comprehensive solution addressing all three key aspects of your issue:

OAuth2 Token Expiration (60-minute limit): Implement automatic token refresh in your PowerShell script. Before each major API operation, check if the current token is within 5 minutes of expiration and refresh proactively:

$tokenExpiry = (Get-Date).AddMinutes(55)
if ((Get-Date) -gt $tokenExpiry) {
    $newToken = Invoke-RestMethod -Uri $refreshEndpoint
}

Store both access and refresh tokens. The refresh token allows you to obtain new access tokens without re-authentication, which is critical for unattended execution.

Unattended Scheduler Execution: The key difference between manual and scheduled runs is the authentication context. Your scheduled task needs to:

  1. Use a dedicated service principal registered in Azure AD with appropriate API permissions (Application type, not Delegated)
  2. Store the client secret in Windows Credential Manager or Azure Key Vault, not in the script
  3. Implement proper logging to track token acquisition and refresh events

Modify your Task Scheduler task to run with “Run whether user is logged on or not” option, and ensure the service account has “Log on as a batch job” rights.

Manual Success vs Scheduled Failure: This discrepancy typically occurs because:

  • Manual runs use your interactive login credentials which may have different token lifetime policies
  • Scheduled runs use service principal authentication which has strict 60-minute access token limits
  • Your manual session might be implicitly refreshing tokens through cached credentials

To resolve, restructure your script:

  1. Initialize with proper service principal authentication (client credentials flow)
  2. Implement a token refresh function that triggers at the 50-minute mark
  3. Add retry logic for API calls that fail with 401 errors, attempting token refresh before retry
  4. Consider batch segmentation as suggested earlier - process cost accounting entries in 45-minute chunks, getting fresh tokens between segments

For D365 Finance 10.0.41 specifically, ensure you’re using the correct OAuth endpoints:

  • Token: `https://login.microsoftonline.com/{tenantId}/oauth2/v2.0/token
  • Refresh: Same endpoint with `grant_type=refresh_token Implement comprehensive error logging to capture token lifecycle events. This will help you verify that token refresh is working correctly in the scheduled context. Your script should log: initial token acquisition, refresh attempts, token expiration times, and any authentication failures with timestamps.

Test the modified script by running it manually with the same service account credentials used by Task Scheduler. This eliminates the variable of different authentication contexts and confirms your token refresh logic works independently of the scheduling mechanism.

Finally, consider implementing a monitoring alert that triggers if the batch job fails due to authentication errors, so you can quickly identify if token refresh logic needs adjustment as Azure AD policies evolve.

This draft is based on general Microsoft Dynamics 365 knowledge. It has not been verified against your specific version and environment. Practitioners: verify the steps and share your experience below.

3

I’ve seen this exact scenario. The core issue is that your initial OAuth token has a 60-minute lifespan, and your script doesn’t implement token refresh logic. When running manually, you might be generating a fresh token closer to execution time, but scheduled tasks use a token that may already be partially aged.

You need to implement a refresh token mechanism. Store the refresh token securely and have your script check token expiration before each API call, or proactively refresh at the 50-minute mark.

4

Thanks for the pointer. I’m using PowerShell for the automation. Are there specific D365 Finance API endpoints for token refresh, or should I be looking at Azure AD token refresh patterns? Our current script just gets a token at the start and uses it throughout.

sureshdev 5

Tested this on our D365 F&O batch import pipeline—the proactive 55-minute PowerShell token refresh check eliminated all mid-job OAuth2 expiration failures completely.

6

You’ll want to use Azure AD’s token refresh endpoint. When you initially authenticate, you receive both an access token and a refresh token. The access token expires in 60 minutes, but the refresh token is valid for much longer (typically 90 days for service accounts).

In your PowerShell script, add logic to track token issue time and proactively request a new access token using the refresh token before the 60-minute window expires. This way your long-running batch process maintains valid authentication throughout its 75-90 minute runtime without manual intervention.

7

Another approach: break your batch into smaller chunks that complete within the 60-minute window. Process data in segments, getting a fresh token for each segment. This also gives you better error recovery - if one segment fails, you don’t lose the entire batch.

For cost accounting imports, you could segment by date range or department. Each segment runs as a separate API session with its own token lifecycle.

8

I’d also verify your service account’s token policies in Azure AD. Sometimes organizational policies enforce stricter timeout rules for service principals. Check if Conditional Access policies are impacting your scheduled runs differently than interactive sessions. We had a case where location-based policies were blocking datacenter-originated requests but allowing office IPs.

9

One more consideration for scheduled runs: token caching behavior. When you run manually, your local credential cache might be providing seamless token refresh. The scheduled task environment doesn’t have this. Make sure your script explicitly handles the complete OAuth flow including refresh tokens, and doesn’t rely on any cached credentials that won’t exist in the Task Scheduler context.