Veeam restore doesn't work in Morpheus

Infrastructure
1

Hello,

I integrated Veeam B&R, Enterprise Manager to Morpheus. It can perform backup however, restore is failed.
Veeam version is 12 and Morpheus is V.5.5.3.

Below is the error message from the current log. Is there anything I need to look at?

2023-03-10_12:51:33.63305 ''[2023-03-10 12:51:33,760] [RxCachedThreadScheduler-218] INFO  c.m.b.VeeamBackupService - Restoring backupResult com.morpheus.backup.BackupResult : 109 - opts: {containerId=2427, restoreExistingContainer=true}
2023-03-10_12:51:33.76081 ''[2023-03-10 12:51:34,167] [RxCachedThreadScheduler-218] ERROR c.m.b.VeeamBackupService - restoreBackup error
2023-03-10_12:51:34.16912 'java.lang.NullPointerException: Cannot get property 'href' on null object
2023-03-10_12:51:34.16913       at org.codehaus.groovy.runtime.NullObject.getProperty(NullObject.java:60)
2023-03-10_12:51:34.16913       at org.codehaus.groovy.runtime.InvokerHelper.getProperty(InvokerHelper.java:194)
2023-03-10_12:51:34.16913       at org.codehaus.groovy.runtime.callsite.NullCallSite.getProperty(NullCallSite.java:46)
2023-03-10_12:51:34.16914       at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callGetProperty(AbstractCallSite.java:329)
2023-03-10_12:51:34.16914       at com.morpheus.backup.VeeamBackupUtility.restoreVM(VeeamBackupUtility.groovy:1566)
2023-03-10_12:51:34.16915       at com.morpheus.backup.VeeamBackupUtility$restoreVM$26.call(Unknown Source)
2023-03-10_12:51:34.16915       at com.morpheus.backup.VeeamBackupService.restoreBackup(VeeamBackupService.groovy:910)
2023-03-10_12:51:34.16915       at com.morpheus.backup.BackupInterface$restoreBackup$1.call(Unknown Source)
2023-03-10_12:51:34.16916       at com.morpheus.backup.BackupRestoreService.restoreBackup(BackupRestoreService.groovy:310)
2023-03-10_12:51:34.16916       at com.morpheus.backup.BackupRestoreService$restoreBackup$4.callCurrent(Unknown Source)
2023-03-10_12:51:34.16918       at com.morpheus.backup.BackupRestoreService.restoreContainerBackup(BackupRestoreService.groovy:237)
2023-03-10_12:51:34.16918       at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
2023-03-10_12:51:34.16919       at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
2023-03-10_12:51:34.16919       at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
2023-03-10_12:51:34.16919       at java.base/java.lang.reflect.Method.invoke(Unknown Source)
2023-03-10_12:51:34.16920       at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:107)
2023-03-10_12:51:34.16920       at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:323)
2023-03-10_12:51:34.16920       at org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:351)
2023-03-10_12:51:34.16921       at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.callCurrent(PogoMetaClassSite.java:61)
2023-03-10_12:51:34.16921       at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:194)
2023-03-10_12:51:34.16921       at com.morpheus.backup.BackupRestoreService$_restoreInstanceBackup_closure6.doCall(BackupRestoreService.groovy:220)
2023-03-10_12:51:34.16923       at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
2023-03-10_12:51:34.16924       at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
2023-03-10_12:51:34.16925       at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
2023-03-10_12:51:34.16925       at java.base/java.lang.reflect.Method.invoke(Unknown Source)
2023-03-10_12:51:34.16925       at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:107)
2023-03-10_12:51:34.16926       at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:323)
2023-03-10_12:51:34.16926       at org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:263)
2023-03-10_12:51:34.16926       at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1035)
2023-03-10_12:51:34.16927       at groovy.lang.Closure.call(Closure.java:412)
2023-03-10_12:51:34.16927       at groovy.lang.Closure.call(Closure.java:428)
2023-03-10_12:51:34.16927       at org.codehaus.groovy.runtime.DefaultGroovyMethods.each(DefaultGroovyMethods.java:2318)
2023-03-10_12:51:34.16928       at org.codehaus.groovy.runtime.DefaultGroovyMethods.each(DefaultGroovyMethods.java:2303)
2023-03-10_12:51:34.16928       at org.codehaus.groovy.runtime.DefaultGroovyMethods.each(DefaultGroovyMethods.java:2344)
2023-03-10_12:51:34.16929       at org.codehaus.groovy.runtime.dgm$200.invoke(Unknown Source)
2023-03-10_12:51:34.16930       at org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite$PojoMetaMethodSiteNoUnwrapNoCoerce.invoke(PojoMetaMethodSite.java:247)
2023-03-10_12:51:34.16930       at org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite.call(PojoMetaMethodSite.java:56)
2023-03-10_12:51:34.16931       at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:139)
2023-03-10_12:51:34.16931       at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callSafe(AbstractCallSite.java:96)
2023-03-10_12:51:34.16932       at com.morpheus.backup.BackupRestoreService.restoreInstanceBackup(BackupRestoreService.groovy:219)
2023-03-10_12:51:34.16932       at com.morpheus.backup.BackupRestoreService$restoreInstanceBackup$3.call(Unknown Source)
2023-03-10_12:51:34.16932       at com.morpheus.InstanceService.restoreInstance(InstanceService.groovy:4292)
2023-03-10_12:51:34.16933       at com.morpheus.InstanceService$restoreInstance$91.call(Unknown Source)
2023-03-10_12:51:34.16933       at com.morpheus.InstanceActionService$_onInstanceRestore_closure18$_closure55.doCall(InstanceActionService.groovy:662)
2023-03-10_12:51:34.16933       at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
2023-03-10_12:51:34.16934       at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
2023-03-10_12:51:34.16935       at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
2023-03-10_12:51:34.16936       at java.base/java.lang.reflect.Method.invoke(Unknown Source)
2023-03-10_12:51:34.16936       at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:107)
2023-03-10_12:51:34.16936       at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:323)
2023-03-10_12:51:34.16937       at org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:263)
2023-03-10_12:51:34.16937       at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1035)
2023-03-10_12:51:34.16937       at groovy.lang.Closure.call(Closure.java:412)
2023-03-10_12:51:34.16938       at groovy.lang.Closure.call(Closure.java:428)
2023-03-10_12:51:34.16938       at org.grails.orm.hibernate.GrailsHibernateTemplate.doExecute(GrailsHibernateTemplate.java:297)
2023-03-10_12:51:34.16939       at org.grails.orm.hibernate.GrailsHibernateTemplate.execute(GrailsHibernateTemplate.java:241)
2023-03-10_12:51:34.16939       at org.grails.orm.hibernate.GrailsHibernateTemplate.executeWithNewSession(GrailsHibernateTemplate.java:153)
2023-03-10_12:51:34.16939       at org.grails.orm.hibernate.AbstractHibernateDatastore.withNewSession(AbstractHibernateDatastore.java:360)
2023-03-10_12:51:34.16940       at org.grails.orm.hibernate.AbstractHibernateGormStaticApi.withNewSession(AbstractHibernateGormStaticApi.groovy:78)
2023-03-10_12:51:34.16941       at org.grails.datastore.gorm.GormEntity$Trait$Helper.withNewSession(GormEntity.groovy:1027)
2023-03-10_12:51:34.16942       at org.grails.datastore.gorm.GormEntity$Trait$Helper$withNewSession.call(Unknown Source)
2023-03-10_12:51:34.16942       at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:47)
2023-03-10_12:51:34.16943       at org.grails.datastore.gorm.GormEntity$Trait$Helper$withNewSession.call(Unknown Source)
2023-03-10_12:51:34.16943       at com.morpheus.Instance.withNewSession(Instance.groovy)
2023-03-10_12:51:34.16943       at com.morpheus.Instance$withNewSession$8.call(Unknown Source)
2023-03-10_12:51:34.16944       at com.morpheus.InstanceActionService$_onInstanceRestore_closure18.doCall(InstanceActionService.groovy:657)
2023-03-10_12:51:34.16944       at com.morpheus.InstanceActionService$_onInstanceRestore_closure18.doCall(InstanceActionService.groovy)
2023-03-10_12:51:34.16944       at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
2023-03-10_12:51:34.16945       at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
2023-03-10_12:51:34.16945       at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
2023-03-10_12:51:34.16947       at java.base/java.lang.reflect.Method.invoke(Unknown Source)
2023-03-10_12:51:34.16947       at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:107)
2023-03-10_12:51:34.16947       at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:323)
2023-03-10_12:51:34.16948       at org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:263)
2023-03-10_12:51:34.16948       at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1035)
2023-03-10_12:51:34.16948       at groovy.lang.Closure.call(Closure.java:412)
2023-03-10_12:51:34.16949       at groovy.lang.Closure.call(Closure.java:406)
2023-03-10_12:51:34.16949       at org.grails.async.factory.rxjava2.RxPromise$_closure1.doCall(RxPromise.groovy:42)
2023-03-10_12:51:34.16950       at jdk.internal.reflect.GeneratedMethodAccessor638.invoke(Unknown Source)
2023-03-10_12:51:34.16950       at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
2023-03-10_12:51:34.16950       at java.base/java.lang.reflect.Method.invoke(Unknown Source)
2023-03-10_12:51:34.16951       at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:107)
2023-03-10_12:51:34.16951       at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:323)
2023-03-10_12:51:34.16951       at org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:263)
2023-03-10_12:51:34.16952       at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1035)
2023-03-10_12:51:34.16953       at groovy.lang.Closure.call(Closure.java:412)
2023-03-10_12:51:34.16954       at org.codehaus.groovy.runtime.ConvertedClosure.invokeCustom(ConvertedClosure.java:50)
2023-03-10_12:51:34.16954       at org.codehaus.groovy.runtime.ConversionHandler.invoke(ConversionHandler.java:110)
2023-03-10_12:51:34.16954       at com.sun.proxy.$Proxy213.subscribe(Unknown Source)
2023-03-10_12:51:34.16955       at io.reactivex.internal.operators.single.SingleCreate.subscribeActual(SingleCreate.java:39)
2023-03-10_12:51:34.16955       at io.reactivex.Single.subscribe(Single.java:3666)
2023-03-10_12:51:34.16955       at io.reactivex.internal.operators.single.SingleSubscribeOn$SubscribeOnObserver.run(SingleSubscribeOn.java:89)
2023-03-10_12:51:34.16956       at io.reactivex.Scheduler$DisposeTask.run(Scheduler.java:608)
2023-03-10_12:51:34.16956       at io.reactivex.internal.schedulers.ScheduledRunnable.run(ScheduledRunnable.java:66)
2023-03-10_12:51:34.16957       at io.reactivex.internal.schedulers.ScheduledRunnable.call(ScheduledRunnable.java:57)
2023-03-10_12:51:34.16957       at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
2023-03-10_12:51:34.16957       at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source)
2023-03-10_12:51:34.16958       at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
2023-03-10_12:51:34.16959       at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
2023-03-10_12:51:34.16960       at java.base/java.lang.Thread.run(Unknown Source)
2023-03-10_12:51:34.18988 '[2023-03-10 12:51:34,189] [http-nio-127.0.0.1-8080-exec-224] INFO

Thanks,
Aung

2

It looks like Veeam made changes that breaks the Morpheus integration in Veeam 12. That release just came out a month ago. If you are looking to use this feature in Morpheus I would recommend running Veeam 11 until Morpheus adds support for Veeam 12. From my testing it looks like just the restores doesn’t work with Morpheus and Veeam 12.

3

Thanks for confirmation @rboyd. Will consider using v11.

Regards,
Aung

4

Hi Team,

I’ve integrated veeam to Morpheus and able to take VM backups successfully, Morpheus API able to trigger backup job in veeam.

Per Morpheus documentation when I try to perform restore it doesn’t work, Restore status under backups > history shows inprogress only and doesn’t change even after 36-48 hours. Have tried removing and reintegrating veeam to Morpheus but no change in behavior when we try to perform restore.

Morpheus version 6.0.3
Veeam version: 12

Unfortunately unable to get veeam version 11 for testing.

Any input/help is greatly appreciated.

Regards
Suresh Thakur

5

Hi Suresh,

v12 is currently not supported in Morpheus. As Bob mentioned, v12 of Veeam has changed the calls for restore and they no longer work in Morpheus. All other functions appear to be working other than restores. This will be available in Morpheus v6.2.7 which should be available soon.

Thanks
Deepti