virt-manager is a gtk interface that connects to libvirt. If you have networking problems, make sure you have the tun module loaded. To run virt-manager as a normal user, you need to configure libvirt to have the correct permissions. Uncomment the unix_sock_group and unix_sock_rw_perms in /etc/libvirt/libvirtd.conf and add the user to the group libvirt uses.