To override default colors of xfce workspace switcher create a css file under ~/.config/gtk-3.0/ like ~/.config/gtk-3.0/gtk.css.

When using the normal pager mode (without numbers), you can use the following - it doesn't work with numbers - :

wnck-pager { background-color: #000000; }  
wnck-pager:selected { background-color: #0000ff; }  
wnck-pager:hover { background-color: #333; }

When pager mode is numbers From Reddit

.xfce4-panel widget grid button.flat.toggle  
{  
color: #F0F8FF;  
background-color: #333;  
min-width: 24px;  
border-style: none;  
}

.xfce4-panel widget grid button.flat.toggle:hover,  
.xfce4-panel widget grid button.flat.toggle:checked  
{  
color: #000;  
background-color: #48D1CC;  
}

To troubleshoot or try things out instantly, quit xfce4 panel process and start it with gtk inspector:

xfce4-panel -q  
GTK_DEBUG=interactive xfce4-panel

Other References:
- https://docs.xfce.org/xfce/xfce4-panel/theming
- https://www.reddit.com/r/xfce/comments/cuzthl/howtocustomizethecolorofthe_workspace/ey28i0y/