In general, classes derived from wxWindow must dynamically allocated with new and deleted with delete. If you delete a window, all of its children and descendants will be automatically deleted, so you don't need to delete these descendants explicitly.
When deleting a frame or dialog, use Destroy rather than delete so that the wxWidgets delayed deletion can take effect. This waits until idle time (when all messages have been processed) to actually delete the window, to avoid problems associated with the GUI sending events to deleted windows.
Don't create a window on the stack, because this will interfere with delayed deletion.
If you decide to allocate a C++ array of objects (such as wxBitmap) that may be cleaned up by wxWidgets, make sure you delete the array explicitly before wxWidgets has a chance to do so on exit, since calling delete on array members will cause memory problems.
wxColour can be created statically: it is not automatically cleaned up and is unlikely to be shared between other objects; it is lightweight enough for copies to be made.
Beware of deleting objects such as a wxPen or wxBitmap if they are still in use. Windows is particularly sensitive to this: so make sure you make calls like wxDC::SetPen(wxNullPen) or wxDC::SelectObject(wxNullBitmap) before deleting a drawing object that may be in use. Code that doesn't do this will probably work fine on some platforms, and then fail under Windows.