Merge pull request #2 from pydata/master

Update from original

Author: danieljl

doc/source/groupby.rst

@@ -784,11 +784,11 @@ will be (silently) dropped. Thus, this does not pose any problems:
 
    df.groupby('A').std()
 
-NA group handling
-~~~~~~~~~~~~~~~~~
+NA and NaT group handling
+~~~~~~~~~~~~~~~~~~~~~~~~~
 
-If there are any NaN values in the grouping key, these will be automatically
-excluded. So there will never be an "NA group". This was not the case in older
+If there are any NaN or NaT values in the grouping key, these will be automatically
+excluded. So there will never be an "NA group" or "NaT group". This was not the case in older
 versions of pandas, but users were generally discarding the NA group anyway
 (and supporting it was an implementation headache).
 

doc/source/whatsnew.rst

@@ -18,7 +18,7 @@ What's New
 
 These are new features and improvements of note in each release.
 
-.. include:: whatsnew/v0.17.0.txt
+.. include:: whatsnew/v0.16.2.txt
 
 .. include:: whatsnew/v0.16.1.txt
 

doc/source/whatsnew/v0.16.2.txt

@@ -0,0 +1,85 @@
+.. _whatsnew_0162:
+
+v0.16.2 (June 12, 2015)
+-----------------------
+
+This is a minor bug-fix release from 0.16.1 and includes a a large number of
+bug fixes along several new features, enhancements, and performance improvements.
+We recommend that all users upgrade to this version.
+
+Highlights include:
+
+Check the :ref:`API Changes <whatsnew_0162.api>` before updating.
+
+.. contents:: What's new in v0.16.2
+    :local:
+    :backlinks: none
+
+.. _whatsnew_0162.enhancements:
+
+New features
+~~~~~~~~~~~~
+
+.. _whatsnew_0162.enhancements.other:
+
+Other enhancements
+^^^^^^^^^^^^^^^^^^
+
+.. _whatsnew_0162.api:
+
+Backwards incompatible API changes
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+.. _whatsnew_0162.api_breaking:
+
+.. _whatsnew_0162.api_breaking.other:
+
+Other API Changes
+^^^^^^^^^^^^^^^^^
+
+- ``Holiday`` now raises ``NotImplementedError`` if both ``offset`` and ``observance`` are used in constructor. (:issue:`102171`)
+
+.. _whatsnew_0162.performance:
+
+Performance Improvements
+~~~~~~~~~~~~~~~~~~~~~~~~
+
+- Improved ``Series.resample`` performance with dtype=datetime64[ns] (:issue:`7754`)
+
+.. _whatsnew_0162.bug_fixes:
+
+Bug Fixes
+~~~~~~~~~
+
+- Bug where read_hdf store.select modifies the passed columns list when
+  multi-indexed (:issue:`7212`)
+- Bug in ``Categorical`` repr with ``display.width`` of ``None`` in Python 3 (:issue:`10087`)
+
+
+- Bug in ``mean()`` where integer dtypes can overflow (:issue:`10172`)
+- Bug where Panel.from_dict does not set dtype when specified (:issue:`10058`)
+- Bug in ``Index.union`` raises ``AttributeError`` when passing array-likes. (:issue:`10149`)
+- Bug in ``Timestamp``'s' ``microsecond``, ``quarter``, ``dayofyear``, ``week`` and ``daysinmonth`` properties return ``np.int`` type, not built-in ``int``. (:issue:`10050`)
+- Bug in ``NaT`` raises ``AttributeError`` when accessing to ``daysinmonth``, ``dayofweek`` properties. (:issue:`10096`)
+
+
+- Bug in getting timezone data with ``dateutil`` on various platforms ( :issue:`9059`, :issue:`8639`, :issue:`9663`, :issue:`10121`)
+- Bug in display datetimes with mixed frequencies uniformly; display 'ms' datetimes to the proper precision. (:issue:`10170`)
+
+- Bung in ``Series`` arithmetic methods may incorrectly hold names (:issue:`10068`)
+
+- Bug in ``DatetimeIndex`` and ``TimedeltaIndex`` names are lost after timedelta arithmetics ( :issue:`9926`)
+
+
+- Bug in `Series.plot(label="LABEL")` not correctly setting the label (:issue:`10119`)
+
+- Bug in `plot` not defaulting to matplotlib `axes.grid` setting (:issue:`9792`)
+
+- Bug in ``Series.align`` resets ``name`` when ``fill_value`` is specified (:issue:`10067`)
+- Bug in ``SparseSeries.abs`` resets ``name`` (:issue:`10241`)
+
+
+- Bug in GroupBy.get_group raises ValueError when group key contains NaT (:issue:`6992`)
+
+
+- Bug where infer_freq infers timerule (WOM-5XXX) unsupported by to_offset (:issue:`9425`)

doc/source/whatsnew/v0.17.0.txt

@@ -1,9 +1,9 @@
 .. _whatsnew_0170:
 
-v0.17.0 (July ??, 2015)
+v0.17.0 (July 31, 2015)
 -----------------------
 
-This is a major release from 0.16.1 and includes a small number of API changes, several new features,
+This is a major release from 0.16.2 and includes a small number of API changes, several new features,
 enhancements, and performance improvements along with a large number of bug fixes. We recommend that all
 users upgrade to this version.
 
@@ -53,29 +53,7 @@ Removal of prior version deprecations/changes
 Performance Improvements
 ~~~~~~~~~~~~~~~~~~~~~~~~
 
-- Improved ``Series.resample`` performance with dtype=datetime64[ns] (:issue:`7754`)
-
 .. _whatsnew_0170.bug_fixes:
 
 Bug Fixes
 ~~~~~~~~~
-
-- Bug where read_hdf store.select modifies the passed columns list when
-  multi-indexed (:issue:`7212`)
-- Bug in ``Categorical`` repr with ``display.width`` of ``None`` in Python 3 (:issue:`10087`)
-
-
-- Bug where Panel.from_dict does not set dtype when specified (:issue:`10058`)
-- Bug in ``Timestamp``'s' ``microsecond``, ``quarter``, ``dayofyear``, ``week`` and ``daysinmonth`` properties return ``np.int`` type, not built-in ``int``. (:issue:`10050`)
-- Bug in ``NaT`` raises ``AttributeError`` when accessing to ``daysinmonth``, ``dayofweek`` properties. (:issue:`10096`)
-
-- Bug in getting timezone data with ``dateutil`` on various platforms ( :issue:`9059`, :issue:`8639`, :issue:`9663`, :issue:`10121`)
-- Bug in display datetimes with mixed frequencies uniformly; display 'ms' datetimes to the proper precision. (:issue:`10170`)
-
-
-- Bug in ``DatetimeIndex`` and ``TimedeltaIndex`` names are lost after timedelta arithmetics ( :issue:`9926`)
-
-- Bug in `Series.plot(label="LABEL")` not correctly setting the label (:issue:`10119`)
-- Bug in `plot` not defaulting to matplotlib `axes.grid` setting (:issue:`9792`)
-
-

pandas/core/generic.py

@@ -3365,11 +3365,10 @@ def _align_series(self, other, join='outer', axis=None, level=None,
                                                          level=level,
                                                          return_indexers=True)
 
-            left_result = self._reindex_indexer(join_index, lidx, copy)
-            right_result = other._reindex_indexer(join_index, ridx, copy)
+            left = self._reindex_indexer(join_index, lidx, copy)
+            right = other._reindex_indexer(join_index, ridx, copy)
 
         else:
-
             # one has > 1 ndim
             fdata = self._data
             if axis == 0:
@@ -3399,23 +3398,19 @@ def _align_series(self, other, join='outer', axis=None, level=None,
             if copy and fdata is self._data:
                 fdata = fdata.copy()
 
-            left_result = DataFrame(fdata)
+            left = DataFrame(fdata)
 
             if ridx is None:
-                right_result = other
+                right = other
             else:
-                right_result = other.reindex(join_index, level=level)
+                right = other.reindex(join_index, level=level)
 
         # fill
         fill_na = notnull(fill_value) or (method is not None)
         if fill_na:
-            return (left_result.fillna(fill_value, method=method, limit=limit,
-                                       axis=fill_axis),
-                    right_result.fillna(fill_value, method=method,
-                                        limit=limit))
-        else:
-            return (left_result.__finalize__(self),
-                    right_result.__finalize__(other))
+            left = left.fillna(fill_value, method=method, limit=limit, axis=fill_axis)
+            right = right.fillna(fill_value, method=method, limit=limit)
+        return (left.__finalize__(self), right.__finalize__(other))
 
     _shared_docs['where'] = ("""
         Return an object of same shape as self and whose corresponding

pandas/core/groupby.py

@@ -426,7 +426,11 @@ def convert(key, s):
                 return Timestamp(key).asm8
             return key
 
-        sample = next(iter(self.indices))
+        if len(self.indices) > 0:
+            sample = next(iter(self.indices))
+        else:
+            sample = None       # Dummy sample
+
         if isinstance(sample, tuple):
             if not isinstance(name, tuple):
                 msg = ("must supply a tuple to get_group with multiple"

pandas/core/index.py

@@ -580,8 +580,18 @@ def to_datetime(self, dayfirst=False):
             return DatetimeIndex(self.values)
 
     def _assert_can_do_setop(self, other):
+        if not com.is_list_like(other):
+            raise TypeError('Input must be Index or array-like')
         return True
 
+    def _convert_can_do_setop(self, other):
+        if not isinstance(other, Index):
+            other = Index(other, name=self.name)
+            result_name = self.name
+        else:
+            result_name = self.name if self.name == other.name else None
+        return other, result_name
+
     @property
     def nlevels(self):
         return 1
@@ -1364,16 +1374,14 @@ def union(self, other):
         -------
         union : Index
         """
-        if not hasattr(other, '__iter__'):
-            raise TypeError('Input must be iterable.')
+        self._assert_can_do_setop(other)
+        other = _ensure_index(other)
 
         if len(other) == 0 or self.equals(other):
             return self
 
         if len(self) == 0:
-            return _ensure_index(other)
-
-        self._assert_can_do_setop(other)
+            return other
 
         if not is_dtype_equal(self.dtype,other.dtype):
             this = self.astype('O')
@@ -1439,11 +1447,7 @@ def intersection(self, other):
         -------
         intersection : Index
         """
-        if not hasattr(other, '__iter__'):
-            raise TypeError('Input must be iterable!')
-
         self._assert_can_do_setop(other)
-
         other = _ensure_index(other)
 
         if self.equals(other):
@@ -1492,18 +1496,12 @@ def difference(self, other):
 
         >>> index.difference(index2)
         """
-
-        if not hasattr(other, '__iter__'):
-            raise TypeError('Input must be iterable!')
+        self._assert_can_do_setop(other)
 
         if self.equals(other):
             return Index([], name=self.name)
 
-        if not isinstance(other, Index):
-            other = np.asarray(other)
-            result_name = self.name
-        else:
-            result_name = self.name if self.name == other.name else None
+        other, result_name = self._convert_can_do_setop(other)
 
         theDiff = sorted(set(self) - set(other))
         return Index(theDiff, name=result_name)
@@ -1517,7 +1515,7 @@ def sym_diff(self, other, result_name=None):
         Parameters
         ----------
 
-        other : array-like
+        other : Index or array-like
         result_name : str
 
         Returns
@@ -1545,13 +1543,10 @@ def sym_diff(self, other, result_name=None):
         >>> idx1 ^ idx2
         Int64Index([1, 5], dtype='int64')
         """
-        if not hasattr(other, '__iter__'):
-            raise TypeError('Input must be iterable!')
-
-        if not isinstance(other, Index):
-            other = Index(other)
-            result_name = result_name or self.name
-
+        self._assert_can_do_setop(other)
+        other, result_name_update = self._convert_can_do_setop(other)
+        if result_name is None:
+            result_name = result_name_update
         the_diff = sorted(set((self.difference(other)).union(other.difference(self))))
         return Index(the_diff, name=result_name)
 
@@ -5460,12 +5455,11 @@ def union(self, other):
         >>> index.union(index2)
         """
         self._assert_can_do_setop(other)
+        other, result_names = self._convert_can_do_setop(other)
 
         if len(other) == 0 or self.equals(other):
             return self
 
-        result_names = self.names if self.names == other.names else None
-
         uniq_tuples = lib.fast_unique_multiple([self.values, other.values])
         return MultiIndex.from_arrays(lzip(*uniq_tuples), sortorder=0,
                                       names=result_names)
@@ -5483,12 +5477,11 @@ def intersection(self, other):
         Index
         """
         self._assert_can_do_setop(other)
+        other, result_names = self._convert_can_do_setop(other)
 
         if self.equals(other):
             return self
 
-        result_names = self.names if self.names == other.names else None
-
         self_tuples = self.values
         other_tuples = other.values
         uniq_tuples = sorted(set(self_tuples) & set(other_tuples))
@@ -5509,18 +5502,10 @@ def difference(self, other):
         diff : MultiIndex
         """
         self._assert_can_do_setop(other)
+        other, result_names = self._convert_can_do_setop(other)
 
-        if not isinstance(other, MultiIndex):
-            if len(other) == 0:
+        if len(other) == 0:
                 return self
-            try:
-                other = MultiIndex.from_tuples(other)
-            except:
-                raise TypeError('other must be a MultiIndex or a list of'
-                                ' tuples')
-            result_names = self.names
-        else:
-            result_names = self.names if self.names == other.names else None
 
         if self.equals(other):
             return MultiIndex(levels=[[]] * self.nlevels,
@@ -5537,15 +5522,30 @@ def difference(self, other):
             return MultiIndex.from_tuples(difference, sortorder=0,
                                           names=result_names)
 
-    def _assert_can_do_setop(self, other):
-        pass
-
     def astype(self, dtype):
         if not is_object_dtype(np.dtype(dtype)):
             raise TypeError('Setting %s dtype to anything other than object '
                             'is not supported' % self.__class__)
         return self._shallow_copy()
 
+    def _convert_can_do_setop(self, other):
+        result_names = self.names
+
+        if not hasattr(other, 'names'):
+            if len(other) == 0:
+                other = MultiIndex(levels=[[]] * self.nlevels,
+                                   labels=[[]] * self.nlevels,
+                                   verify_integrity=False)
+            else:
+                msg = 'other must be a MultiIndex or a list of tuples'
+                try:
+                    other = MultiIndex.from_tuples(other)
+                except:
+                    raise TypeError(msg)
+        else:
+            result_names = self.names if self.names == other.names else None
+        return other, result_names
+
     def insert(self, loc, item):
         """
         Make new MultiIndex inserting new item at location