We are getting jobs frozen when updating ADS Groups in One of the domain. We have 3 domains in OneIM... two of them have no issues provisioning any changes. One domain is having issues on ADS_ADSGroup_Update and sometimes on ADS_ADSGroup_Insert. below is the error i am getting. We tried to make a change thru SyncEditor and going to Browse for that group and the change was successful to AD.
ErrorMessages = (2019-04-03 13:54:59.630) [1777018] Error executing synchronization project (Active Directory Domain (DC=tmk,DC=ent,DC=lc))'s workflow (Provisioning - tmk). [1777124] Error executing projection step (group) of projection configuration (Provisioning - tmk (Provisioning)). [1777219] Error executing synchronization step (group)! [1777004] Method (Update object (Update)) could not be executed successfully. [2226012] Error committing object CN=RG_780ITOP,OU=RoleGroups,OU=AllGroups,DC=tmk,DC=ent,DC=lc.(Error: Error committing object ea638b62-383a-47ae-9a6f-37a3df3da188.(Error: Querying GC://MKTMP1DC01.tmk.ent.lc using filter (DistinguishedName=CN=Tonja L. Armstrong,OU=DisabledAccounts,OU=AllUsers,DC=tmk,DC=ent,DC=lc) Querying GC MKTMP1DC01.tmk.ent.lc results in error: The server is not operational. . Querying GC://o2tmp1dc01.tmk.ent.lc using filter (DistinguishedName=CN=Tonja L. Armstrong,OU=DisabledAccounts,OU=AllUsers,DC=tmk,DC=ent,DC=lc) Error executing group_managedby_Set on object CN=RG_780ITOP,OU=RoleGroups,OU=AllGroups,DC=tmk,DC=ent,DC=lc (Error: [System.Reflection.TargetInvocationException] Exception has been thrown by the target of an invocation. [System.Runtime.InteropServices.COMException] The server is not operational. ). Error writing property vrtmanagedBy. The modification will rerun after committing object. 1 write operations on properties were deferred and will be executed now. Property vrtmanagedBy will be set. Querying GC://MKTMP1DC01.tmk.ent.lc using filter (DistinguishedName=CN=Tonja L. Armstrong,OU=DisabledAccounts,OU=AllUsers,DC=tmk,DC=ent,DC=lc) Querying GC MKTMP1DC01.tmk.ent.lc results in error: The server is not operational. . Querying GC://o2tmp1dc01.tmk.ent.lc using filter (DistinguishedName=CN=Tonja L. Armstrong,OU=DisabledAccounts,OU=AllUsers,DC=tmk,DC=ent,DC=lc) Error executing group_managedby_Set on object CN=RG_780ITOP,OU=RoleGroups,OU=AllGroups,DC=tmk,DC=ent,DC=lc (Error: [System.Reflection.TargetInvocationException] Exception has been thrown by the target of an invocation. [System.Runtime.InteropServices.COMException] The server is not operational. ). The server is not operational. Exception has been thrown by the target of an invocation. Error writing one or more properties of object. Object not committed successfully. Retrying using single property commit. Property vrtmanagedBy will be set. Querying GC://MKTMP1DC01.tmk.ent.lc using filter (DistinguishedName=CN=Tonja L. Armstrong,OU=DisabledAccounts,OU=AllUsers,DC=tmk,DC=ent,DC=lc) Querying GC MKTMP1DC01.tmk.ent.lc results in error: The server is not operational. . Querying GC://o2tmp1dc01.tmk.ent.lc using filter (DistinguishedName=CN=Tonja L. Armstrong,OU=DisabledAccounts,OU=AllUsers,DC=tmk,DC=ent,DC=lc) Error executing group_managedby_Set on object CN=RG_780ITOP,OU=RoleGroups,OU=AllGroups,DC=tmk,DC=ent,DC=lc (Error: [System.Reflection.TargetInvocationException] Exception has been thrown by the target of an invocation. [System.Runtime.InteropServices.COMException] The server is not operational. ). The server is not operational. Exception has been thrown by the target of an invocation. )) at VI.Projector.JobComponent.ProjectorComponent._AdHocProjection() at VI.Projector.Database.ProjectorExecutor.Project(ISession session, IProjectionConfiguration configuration, ProjectionOption options, ISystemObject adHocObject, CancellationToken cancellationToken) at VI.Projector.Projection.Extensions.Execute(IProjectorEngine engine, IProjectionConfiguration configuration, ProjectionOption options, ISystemObject adHocObject, CancellationToken cancellationToken) at System.Threading.Tasks.Task.Wait(Int32 millisecondsTimeout, CancellationToken cancellationToken) ---- Start of Inner Exception ---- at VI.Projector.Projection.ProjectorEngine.<ExecuteAsync>d__3.MoveNext() ---- Start of Inner Exception ---- at VI.Projector.Projection.ProjectorEngine.<ExecuteAsync>d__3.MoveNext() ---- Start of Inner Exception ---- at VI.Projector.Projection.ProjectorEngine.<ExecuteAsync>d__3.MoveNext() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) --- End of stack trace from previous location where exception was thrown --- at VI.Projector.Projection.ProjectionStrategy.<ExecuteStepAsync>d__2.MoveNext() ---- Start of Inner Exception ---- at VI.Projector.Projection.ProjectionStrategy.<ExecuteStepAsync>d__2.MoveNext() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) --- End of stack trace from previous location where exception was thrown --- at VI.Projector.Projection.ProjectionStrategyBase.<OnExecuteStepAsync>d__2.MoveNext() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) --- End of stack trace from previous location where exception was thrown --- at VI.Projector.Projection.ProjectionStrategyBase.<_ProcessIntersectionSetAsync>d__17.MoveNext() at System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1.ConfiguredTaskAwaiter.GetResult() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) --- End of stack trace from previous location where exception was thrown --- at VI.Projector.Projection.ProjectionStrategyBase.<_ProcessMappingOfSystemObjectPairsAsync>d__21.MoveNext() at System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1.ConfiguredTaskAwaiter.GetResult() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) --- End of stack trace from previous location where exception was thrown --- at System.Threading.Tasks.Task.Execute() at System.Threading.Tasks.Task`1.InnerInvoke() at VI.Projector.Projection.ProjectionStrategyBase.<>c__DisplayClass21_0.<_ProcessMappingOfSystemObjectPairsAsync>b__9() at VI.Projector.Projection.ProjectionStrategyBase._ExecuteMethods(IProjectionStep step, SystemObjectMatchingSet matchingSet, ISystemConnection connection, ISchemaMethodAssignment[] methodAssignments, IList`1 systemObjectPairs, SystemMappingSide side, ProjectionDirection projectionDirection, Boolean forceReloadObject, Boolean ignoreQuotas) at VI.Projector.Projection.ProjectionStrategyBase._ExecuteMethodNative(ISystemConnection connection, SystemObjectMatchingSet matchingSet, ISchemaMethod method, IEnumerable`1 objectsWithAncestryToExecute, Boolean reloadObjects, Boolean isImport) at VI.Projector.Connection.SystemConnection.ExecuteMethod(ISchemaMethod method, IEnumerable`1 systemObjects, ExecuteMethodOptions option) ---- Start of Inner Exception ---- at VI.Projector.Connection.SystemConnection.ExecuteMethod(ISchemaMethod method, IEnumerable`1 systemObjects, ExecuteMethodOptions option) at VI.Projector.Journal.JournalConnector.OnExecuteMethod(ExecuteMethodRequest request) at VI.Projector.Connection.Connectors.BottleNeckConnector._Redirect[T](Func`1 redirection) at System.Threading.Tasks.Task.Wait(Int32 millisecondsTimeout, CancellationToken cancellationToken) ---- Start of Inner Exception ---- at VI.Projector.Connection.Connectors.BottleNeckConnectorTask.Execute() at VI.Projector.Connection.Connectors.BottleNeckConnector.<>c__DisplayClass12_0.<OnExecuteMethod>b__0() at VI.Projector.Connection.SystemConnector.ExecuteMethod(ExecuteMethodRequest request) at VI.Projector.Connection.SystemConnector.CommitObject(CommitObjectRequest request) at VI.Projector.ADS.ProjectorADSConnector.OnCommitObject(CommitObjectRequest request) at VI.Projector.ProjectorBaseConnector.HandleException(ObjectRequest request, Exception Ex) Synchronization ended with errors. You can find more detailed information in the log from Wednesday, April 3, 2019.
Any and all help would be greatly appreciated.
Thank you,