Reverting some changes in FetchUrl back to see if can get multi-curl and linux to behave, a=chris

Chris Pollett [2016-09-25 05:Sep:th]
Reverting some changes in FetchUrl back to see if can get multi-curl and linux to behave, a=chris
Filename
src/library/FetchUrl.php
diff --git a/src/library/FetchUrl.php b/src/library/FetchUrl.php
index 5da10c174..c98bf7de2 100755
--- a/src/library/FetchUrl.php
+++ b/src/library/FetchUrl.php
@@ -317,7 +317,16 @@ class FetchUrl implements CrawlConstants
                             strtolower(trim($type_parts[0]));
                     }
                 }
-                curl_multi_remove_handle($agent_handler, $sites[$i][0]);
+                 /*
+                   Ideally should have line for all requests
+                   However, this seems to cause curl to sometimes crash
+                   by trying to free stuff twice on some linux systems
+                   at crawl time. Not having it on other systems causes crashes
+                   at query time
+                */
+                if ($minimal || !stristr(PHP_OS, "LINUX")) {
+                    curl_multi_remove_handle($agent_handler, $sites[$i][0]);
+                }
                 curl_close($sites[$i][0]);
                 if (isset($sites[$i]['ROBOT']) && $sites[$i]['ROBOT']) {
                     if (isset($sites[$i][self::TYPE]) &&
ViewGit