mzy2240/ESA

Value error during object instantiation

mzy2240 opened this issue · 5 comments

mzy2240 commented

error

@blthayer Do you have any idea why this happens? This is a snapshot someone sent to me this morning, he is using a Python 3.9.9 env. Meanwhile, there is another guy encountered this issue as well, he is using Windows 11 with Python 3.8.3. Both have this issues for multiple esa versions.

I cannot reproduce the error with same case (but Python 3.9.7). The latest esa passed all the tests in my two computers. @SmartJingJing123 also has no issues with the latest version.

mzy2240 commented

They tested pycharm, spyder and notebook. All have the same error.

blthayer commented

@mzy2240

When you say you cannot reproduce with the same case, do you mean they sent you the .PWB file and it worked fine for you?

If so, the lack of reproducibility might be related to differences in PowerWorld version.

Here's the GetFieldList method from v1.2.2:

    def GetFieldList(self, ObjectType: str, copy=False) -> pd.DataFrame:
        """Get all fields associated with a given ObjectType.
        :param ObjectType: The type of object for which the fields are
            requested.
        :param copy: Whether or not to return a copy of the DataFrame.
            You may want a copy if you plan to make any modifications.
        :returns: Pandas DataFrame with columns from either
            SAW.FIELD_LIST_COLUMNS or SAW.FIELD_LIST_COLUMNS_OLD,
            depending on the version of PowerWorld Simulator being used.
        `PowerWorld Documentation
        <https://www.powerworld.com/WebHelp/#MainDocumentation_HTML/GetFieldList_Function.htm>`__
        """
        # Get the ObjectType in lower case.
        object_type = ObjectType.lower()

        # Either look up stored DataFrame, or call SimAuto.
        try:
            output = self._object_fields[object_type]
        except KeyError:
            # We haven't looked up fields for this object yet.
            # Call SimAuto.
            result = self._call_simauto('GetFieldList', ObjectType)

            # Place result in a numpy array.
            result_arr = np.array(result)

            # Attempt to map results into a DataFrame using
            # FIELD_LIST_COLUMNS. If that fails, use
            # FIELD_LIST_COLUMNS_OLD.
            try:
                output = pd.DataFrame(result_arr,
                                      columns=self.FIELD_LIST_COLUMNS)
            except ValueError as e:
                # We may be dealing with the older convention.
                # The value error should read something like:
                # "Shape of passed values is (259, 4), indices imply (259, 5)"
                # Confirm via regular expressions.
                exp_base = r'\([0-9]+,\s'
                exp_end = r'{}\)'
                # Get number of columns for new/old lists.
                nf_old = len(self.FIELD_LIST_COLUMNS_OLD)
                nf_new = len(self.FIELD_LIST_COLUMNS)
                # Search the error's arguments.
                r1 = re.search(exp_base + exp_end.format(nf_old), e.args[0])
                r2 = re.search(exp_base + exp_end.format(nf_new), e.args[0])

                # Both results should match, i.e., not be None.
                if (r1 is None) or (r2 is None):
                    raise e

                # If we made it here, use the older columns.
                output = pd.DataFrame(result_arr,
                                      columns=self.FIELD_LIST_COLUMNS_OLD)

            # While it appears PowerWorld gives us the list sorted by
            # internal_field_name, let's make sure it's always sorted.
            output.sort_values(by=['internal_field_name'], inplace=True)

            # Store this for later.
            self._object_fields[object_type] = output

        # Either return a copy or not.
        if copy:
            return output.copy(deep=True)
        else:
            return output

Purely from reading the code and stack trace, I think (though I'm not 100% sure) that we hit the following raise e block:

                # Both results should match, i.e., not be None.
                if (r1 is None) or (r2 is None):
                    raise e

This is essentially re-raising the exception (ValueError) from pandas because the regular expression(s) didn't match.

Take a look at the stack trace provided by the user. The last line reads:
ValueError: Shape of passed values is (396, 6), indices imply (396, 5)

Looking at the code, I expect that len(self.FIELD_LIST_COLUMNS) is 5 and len(self.FIELD_LIST_COLUMNS_OLD is 4. Therefore, only one of r1 or r2 is not None, causing us to raise e.

So it almost seems like PowerWorld is returning data that maps to 6 columns, but we only know how to support returns of 5 or 4 columns.

  • 👍1
mzy2240 commented

PW released version 23 and that might be the reason. I did not expect that tbh. Will check with them on the PW versions. Thanks!

blthayer commented
mzy2240 commented

Starting from November last year, a new field "enterable" has been added to object fields in PW. I will release a new version fixing this very soon.