Now I would like to remove that empty panel which is not having any data points the remove option, 
is working, but again the panel is reproducing when we refresh the dashboard.
Can you please advise how to delete the non-used panel?
You need to remove a panel and save a dashboard.
Did the same thing which you have said, later I refreshed the dashboard then the removed panel again displaying in the dashboard. Thanks!
Check the logs and be sure that you write permission for your dashboard.