In the following example I ran into an issue whereby I was attempting to:
Migrate VMware Virtual Machine (VM) 2012_DC01.BINTRI.AD from ESXi Host-A to ESXi Host-B
The Migration was successful.
However after the VMotion completed, when I attempted to 'ping' my VM host IP Address it showed that there was a Duplicate IP address!
# ping -I ens192 172.16.241.187
PING 172.16.241.187 (172.16.241.187) from 172.16.242.196 ens192: 56(84) bytes of data.
64 bytes from 172.16.241.187: icmp_seq=1 ttl=128 time=0.459 ms
64 bytes from 172.16.241.187: icmp_seq=1 ttl=128 time=0.460 ms (DUP!)
64 bytes from 172.16.241.187: icmp_seq=2 ttl=128 time=0.529 ms
64 bytes from 172.16.241.187: icmp_seq=2 ttl=128 time=0.529 ms (DUP!)
I knew my client (2012_DC01.BINTRI.AD) only had a single ethernet interface.
Initially thinking someone grabbed my IP address on the network, I ran the following to display the MAC address
# arping -c 1 -I ens192 172.16.241.187
ARPING 172.16.241.187 from 172.16.242.51 ens192
Unicast reply from 172.16.241.187 [00:50:56:AF:7A:19] 1.390ms
Unicast reply from 172.16.241.187 [00:50:56:AF:7A:19] 1.432ms
Sent 1 probes (1 broadcast(s))
Received 2 response(s)
This told me that there was the SAME MAC address (00:50:56:AF:7A:19) responding to IP 172.16.241.187
Because I wanted to be doubly sure, I used nmap to tell me the OS type/etc and I was able to verify it is a Windows Server 2012 Client MAC Address (00:50:56:AF:7A:19)
# nmap -FO --system-dns 172.16.241.187
Starting Nmap 6.40 ( http://nmap.org ) at 2022-01-24 16:02 PST
Nmap scan report for dc01.bintri.ad (172.16.241.187)
Host is up (0.00033s latency).
Not shown: 95 closed ports
PORT STATE SERVICE
53/tcp open domain
80/tcp open http
139/tcp open netbios-ssn
445/tcp open microsoft-ds
49152/tcp open unknown
MAC Address: 00:50:56:AF:7A:19 (VMware)
Device type: general purpose
Running: Microsoft Windows 7|2012
OS CPE: cpe:/o:microsoft:windows_7:::ultimate cpe:/o:microsoft:windows_2012
OS details: Microsoft Windows 7 or Windows Server 2012
Network Distance: 1 hop
OS detection performed. Please report any incorrect results at http://nmap.org/submit/ .
Nmap done: 1 IP address (1 host up) scanned in 4.47 seconds
Because I knew MAC Address (00:50:56:AF:7A:19) belonged to (2012_DC01.BINTRI.AD) there was 'something' in ESXi that believed there were 2 instances running!
I powered off VM 2012_DC01.BINTRI.AD using vCenter.
Yet I was still able to ping the IP address.
I went to the source ESXi Host-A and verified that it did not exist.
This told me that something was still resident in memory on either ESXi Host-A, or ESXi Host-B
I ssh'd into ESXi Host-A and ran the following command:
# esxcli vm process list
I looked for any processes open for host 2012_DC01.BINTRI.AD. I saw none.
I then ssh'd into ESXi Host-B and also ran
# esxcli vm process list
Except this time I saw in the output:
# esxcli vm process list
AD_2012_DC01.BINTRI.AD
World ID: 136755
Process ID: 0
VMX Cartel ID: 136754
UUID: 42 2f 8d 75 b6 84 51 78-90 5b 06 84 b2 e4 d5 22
Display Name: AD_2012_DC01.BINTRI.AD
Config File: /vmfs/volumes/d81de9d4-9c2bbc60/AD_2012_BENDC01.BINTRI.AD/AD_2012_BENDC01.BINTRI.AD.vmx
Because I powered off the VM, this told me that there was a process that needed to be stopped.
Therefore I used the 'World ID' and ran the following:
# esxcli vm process kill -t force -w=136755
Re-running esxcli vm process list now showed the process was gone.
I attempted to ping the IP address and this time it did not respond; as expected.
I then powered on the VM, and now when I ping, there is only a single IP Address
# ping 172.16.241.187
PING 172.16.241.187 (172.16.241.187) 56(84) bytes of data.
64 bytes from 172.16.241.187: icmp_seq=1 ttl=128 time=1.68 ms
64 bytes from 172.16.241.187: icmp_seq=2 ttl=128 time=0.348 ms
64 bytes from 172.16.241.187: icmp_seq=3 ttl=128 time=0.606 ms
Welcome
Welcome to my blog. This is my first post and I will be sharing more information soon.