Modifying the tinymce-code is a quick and dirty way to fix this issue because it does not really solve the problem for a long time (update compatibility, etc.).
The proper solution is to alter the silverstripe code to make it remove the editor from tinymces list of active editors before removing the widgets html object from the dom.

You can leave your tiny_mce_src.js as it is and modify the “cms/javascript/WidgetAreaEditor.js” on line 126 like this (or use attached file):

Change

deleteWidget: function(widgetToRemove) {
   // Remove a widget from the used widgets column
   $('WidgetAreaEditor_usedWidgets').removeChild(widgetToRemove);
}

To

deleteWidget: function(widgetToRemove) {
   // Remove the widgets htmleditor areas from tinymces list
   jQuery(widgetToRemove).find('textarea[id]').each(function () {
      var tinyId = jQuery(this).attr('id');
      if (tinyMCE.get(tinyId)) {
         tinyMCE.execCommand('mceFocus', false, tinyId);
         tinyMCE.execCommand('mceRemoveControl', false, tinyId);
      }
   });
   // Remove a widget from the used widgets column
   $('WidgetAreaEditor_usedWidgets').removeChild(widgetToRemove);
}

I will commit this changes to silverstripe to have this problem removed permanently for the future.

I was able to correct the issue with Tiny MCE by applying the fix that FungshuiElephant referred to in this post http://www.silverstripe.org/widgets-2/show/258893?start=0#post261255

I am using SS 2.3.4 and dont have any other widgets in use, so I am not sure if it affects other widgets, but it loads the Tiny MCE HTML editor properly on every visit to the Widget admin area.