vCloud Director | How to re-synchronize inventory of vCenter server with vCD

Basically we need to do this due to stale entries in vCD database. It happens when any object get removed\deleted directly from vCenter while it was being managed by vCD.

Hence vCD don't know if that was deleted and treat it like live object which creates issues in vCD operations. By re-sync inventory, vCD re-read all the inventory of vCenter server and update in database which resolve the issue.

It demands downtime as well. Depending upon your environment it takes few minutes to an hour.

Step 1: Take backup of vCD database.
Step 2: Take backup of vCenter database.

In order to minimize the impact, you should do this on all vCD cells one by one. First quiesce(stop) all the jobs coming on a cell.

Step 3:  Login to the vCloud Director cell via SSH as root and change directory to where the cell-management-tool is located. Below is the command.

  # cd /opt/vmware/vcloud-director/bin

Step 4: Run the tool to verify the job count and active state (true):

  # ./cell-management-tool -u administrator cell -t

#below is the command output

Please enter the administrator password:

Job count = 2

Is Active = true
#End here

Step 5:  Use the quiesce feature to ensure the cell takes no new jobs

  # ./cell-management-tool -u administrator cell -q true

Please enter the administrator password:

Step 6: Run the tool to verify the job count and active state (false):

  # ./cell-management-tool -u administrator cell -t

Please enter the administrator password:

Job count = 0

Is Active = false

Step 7: Restart the cell once the Job count = 0 and Is Active = false:

  # ./cell-management-tool -u administrator cell -s

Please enter the administrator password:

Step 8: Stop the vCloud Director services:

  # service vmware-vcd stop

#Below is the output of above command

Stopping vmware-vcd-watchdog:        [  OK  ]

Stopping vmware-vcd-cell:            [  OK  ]

#End here

****NOTE: You must stop ALL of your cells before proceeding to the queries****  

Step 9: Once above process is carried out on all the cells. Access and connect vCD database and paste below lines

#start here

Delete from task;

delete from audit_event;

delete from jobs where job_id not in(select job_id from job_detail);

update jobs set status = 3 where status = 1;

update last_jobs set status = 3 where status = 1;

delete from busy_object;

delete from QRTZ_SCHEDULER_STATE;

delete from QRTZ_FIRED_TRIGGERS;

delete from QRTZ_PAUSED_TRIGGER_GRPS;

delete from QRTZ_CALENDARS;

delete from QRTZ_TRIGGER_LISTENERS;

delete from QRTZ_BLOB_TRIGGERS;

delete from QRTZ_CRON_TRIGGERS;

delete from QRTZ_SIMPLE_TRIGGERS;

delete from QRTZ_TRIGGERS;

delete from QRTZ_JOB_LISTENERS;

delete from QRTZ_JOB_DETAILS;

delete from compute_resource_inv;

delete from custom_field_manager_inv;

delete from ccr_drs_vm_host_rule_inv;

delete from cluster_compute_resource_inv;

delete from datacenter_inv;

delete from datacenter_network_inv;

delete from datastore_inv;

delete from dv_portgroup_inv;

delete from dv_switch_inv;

delete from folder_inv;

delete from managed_server_inv;

delete from managed_server_datastore_inv;

delete from managed_server_network_inv;

delete from network_inv;

delete from resource_pool_inv;

delete from storage_pod_inv;

delete from task_inv;

delete from task_activity_queue;

delete from activity;

delete from activity_parameters;

delete from failed_cells;

delete from lock_handle;

delete from vm_inv;

delete from property_map;

commit;

#End here

Step 10: Once done, start all the vCD cells services and check vCD portal accessibility. 

Please note: After this operation, you will see errors on all OrgvDCs which is normal, you can ignore and clear those alerts.

Disclaimer: It is destructive process so, it is recommended to do the same in supervision of some experienced person or under guidance of VMware.

No comments |  October 10, 2019