{ "__type": "IngestedDoc", "__tag": 4010, "_content": {}, "_ordered_sections": [], "item_file": null, "item_line": null, "item_type": null, "aliases": [], "example_section_data": { "__type": "Section", "__tag": 4015, "children": [], "title": [], "level": 0, "target": null }, "see_also": [], "signature": null, "references": null, "qa": "user:basics.subclassing", "arbitrary": [ { "__type": "Section", "__tag": 4015, "children": [], "title": [ { "__type": "Text", "__tag": 4046, "value": "Subclassing ndarray" } ], "level": 0, "target": "basics.subclassing" }, { "__type": "Section", "__tag": 4015, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Subclassing ndarray is relatively simple, but it has some complications compared to other Python objects. On this page we explain the machinery that allows you to subclass ndarray, and the implications for implementing a subclass." } ] } ], "title": [ { "__type": "Text", "__tag": 4046, "value": "Introduction" } ], "level": 1, "target": null }, { "__type": "Section", "__tag": 4015, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Subclassing ndarray is complicated by the fact that new instances of ndarray classes can come about in three different ways. These are:" } ] }, { "__type": "BulletList", "__tag": 4053, "ordered": true, "start": 1, "children": [ { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Explicit constructor call - as in " }, { "__type": "InlineCode", "__tag": 4051, "value": "MySubClass(params)" }, { "__type": "Text", "__tag": 4046, "value": ". This is the usual route to Python instance creation." } ] } ] }, { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "View casting - casting an existing ndarray as a given subclass" } ] } ] }, { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "New from template - creating a new instance from a template instance. Examples include returning slices from a subclassed array, creating return types from ufuncs, and copying arrays. See " }, { "__type": "CrossRef", "__tag": 4002, "value": "new-from-template", "reference": { "__type": "LocalRef", "__tag": 4022, "kind": "docs", "path": "user:basics.subclassing" }, "kind": "exists" }, { "__type": "Text", "__tag": 4046, "value": " for more details" } ] } ] } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "The last two are characteristics of ndarrays - in order to support things like array slicing. The complications of subclassing ndarray are due to the mechanisms numpy has to support these latter two routes of instance creation." } ] } ], "title": [ { "__type": "Text", "__tag": 4046, "value": "ndarrays and object creation" } ], "level": 2, "target": null }, { "__type": "Section", "__tag": 4015, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Besides the additional complexities of subclassing a NumPy array, subclasses can run into unexpected behaviour because some functions may convert the subclass to a baseclass and \"forget\" any additional information associated with the subclass. This can result in surprising behavior if you use NumPy methods or functions you have not explicitly tested." } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "On the other hand, compared to other interoperability approaches, subclassing can be useful because many things will \"just work\"." } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "This means that subclassing can be a convenient approach and for a long time it was also often the only available approach. However, NumPy now provides additional interoperability protocols described in \"" }, { "__type": "CrossRef", "__tag": 4002, "value": "Interoperability with NumPy", "reference": { "__type": "LocalRef", "__tag": 4022, "kind": "docs", "path": "user:basics.interoperability" }, "kind": "exists" }, { "__type": "Text", "__tag": 4046, "value": "\". For many use-cases these interoperability protocols may now be a better fit or supplement the use of subclassing." } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Subclassing can be a good fit if:" } ] }, { "__type": "BulletList", "__tag": 4053, "ordered": false, "start": 1, "children": [ { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "you are less worried about maintainability or users other than yourself: Subclass will be faster to implement and additional interoperability can be added \"as-needed\". And with few users, possible surprises are not an issue." } ] } ] }, { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "you do not think it is problematic if the subclass information is ignored or lost silently. An example is " }, { "__type": "InlineCode", "__tag": 4051, "value": "np.memmap" }, { "__type": "Text", "__tag": 4046, "value": " where \"forgetting\" about data being memory mapped cannot lead to a wrong result. An example of a subclass that sometimes confuses users are NumPy's masked arrays. When they were introduced, subclassing was the only approach for implementation. However, today we would possibly try to avoid subclassing and rely only on interoperability protocols." } ] } ] } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Note that also subclass authors may wish to study " }, { "__type": "CrossRef", "__tag": 4002, "value": "Interoperability with NumPy", "reference": { "__type": "LocalRef", "__tag": 4022, "kind": "docs", "path": "user:basics.interoperability" }, "kind": "exists" }, { "__type": "Text", "__tag": 4046, "value": " to support more complex use-cases or work around the surprising behavior." } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "InlineCode", "__tag": 4051, "value": "astropy.units.Quantity" }, { "__type": "Text", "__tag": 4046, "value": " and " }, { "__type": "InlineCode", "__tag": 4051, "value": "xarray" }, { "__type": "Text", "__tag": 4046, "value": " are examples for array-like objects that interoperate well with NumPy. Astropy's " }, { "__type": "InlineCode", "__tag": 4051, "value": "Quantity" }, { "__type": "Text", "__tag": 4046, "value": " is an example which uses a dual approach of both subclassing and interoperability protocols." } ] } ], "title": [ { "__type": "Text", "__tag": 4046, "value": "When to use subclassing" } ], "level": 2, "target": null }, { "__type": "Section", "__tag": 4015, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Emphasis", "__tag": 4047, "children": [ { "__type": "Text", "__tag": 4046, "value": "View casting" } ] }, { "__type": "Text", "__tag": 4046, "value": " is the standard ndarray mechanism by which you take an ndarray of any subclass, and return a view of the array as another (specified) subclass:" } ] }, { "__type": "Code", "__tag": 4050, "value": ">>> import numpy as np\n>>> # create a completely useless ndarray subclass\n>>> class C(np.ndarray): pass\n>>> # create a standard ndarray\n>>> arr = np.zeros((3,))\n>>> # take a view of it, as our useless subclass\n>>> c_arr = arr.view(C)\n>>> type(c_arr)\n", "execution_status": null } ], "title": [ { "__type": "Text", "__tag": 4046, "value": "View casting" } ], "level": 1, "target": "view-casting" }, { "__type": "Section", "__tag": 4015, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "New instances of an ndarray subclass can also come about by a very similar mechanism to " }, { "__type": "CrossRef", "__tag": 4002, "value": "view-casting", "reference": { "__type": "LocalRef", "__tag": 4022, "kind": "docs", "path": "user:basics.subclassing" }, "kind": "exists" }, { "__type": "Text", "__tag": 4046, "value": ", when numpy finds it needs to create a new instance from a template instance. The most obvious place this has to happen is when you are taking slices of subclassed arrays. For example:" } ] }, { "__type": "Code", "__tag": 4050, "value": ">>> v = c_arr[1:]\n>>> type(v) # the view is of type 'C'\n\n>>> v is c_arr # but it's a new instance\nFalse", "execution_status": null }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "The slice is a " }, { "__type": "Emphasis", "__tag": 4047, "children": [ { "__type": "Text", "__tag": 4046, "value": "view" } ] }, { "__type": "Text", "__tag": 4046, "value": " onto the original " }, { "__type": "InlineCode", "__tag": 4051, "value": "c_arr" }, { "__type": "Text", "__tag": 4046, "value": " data. So, when we take a view from the ndarray, we return a new ndarray, of the same class, that points to the data in the original." } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "There are other points in the use of ndarrays where we need such views, such as copying arrays (" }, { "__type": "InlineCode", "__tag": 4051, "value": "c_arr.copy()" }, { "__type": "Text", "__tag": 4046, "value": "), creating ufunc output arrays (see also " }, { "__type": "CrossRef", "__tag": 4002, "value": "array-wrap", "reference": { "__type": "LocalRef", "__tag": 4022, "kind": "docs", "path": "user:basics.subclassing" }, "kind": "exists" }, { "__type": "Text", "__tag": 4046, "value": "), and reducing methods (like " }, { "__type": "InlineCode", "__tag": 4051, "value": "c_arr.mean()" }, { "__type": "Text", "__tag": 4046, "value": ")." } ] } ], "title": [ { "__type": "Text", "__tag": 4046, "value": "Creating new from template" } ], "level": 1, "target": "new-from-template" }, { "__type": "Section", "__tag": 4015, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "These paths both use the same machinery. We make the distinction here, because they result in different input to your methods. Specifically, " }, { "__type": "CrossRef", "__tag": 4002, "value": "view-casting", "reference": { "__type": "LocalRef", "__tag": 4022, "kind": "docs", "path": "user:basics.subclassing" }, "kind": "exists" }, { "__type": "Text", "__tag": 4046, "value": " means you have created a new instance of your array type from any potential subclass of ndarray. " }, { "__type": "CrossRef", "__tag": 4002, "value": "new-from-template", "reference": { "__type": "LocalRef", "__tag": 4022, "kind": "docs", "path": "user:basics.subclassing" }, "kind": "exists" }, { "__type": "Text", "__tag": 4046, "value": " means you have created a new instance of your class from a pre-existing instance, allowing you - for example - to copy across attributes that are particular to your subclass." } ] } ], "title": [ { "__type": "Text", "__tag": 4046, "value": "Relationship of view casting and new-from-template" } ], "level": 1, "target": null }, { "__type": "Section", "__tag": 4015, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "If we subclass ndarray, we need to deal not only with explicit construction of our array type, but also " }, { "__type": "CrossRef", "__tag": 4002, "value": "view-casting", "reference": { "__type": "LocalRef", "__tag": 4022, "kind": "docs", "path": "user:basics.subclassing" }, "kind": "exists" }, { "__type": "Text", "__tag": 4046, "value": " or " }, { "__type": "CrossRef", "__tag": 4002, "value": "new-from-template", "reference": { "__type": "LocalRef", "__tag": 4022, "kind": "docs", "path": "user:basics.subclassing" }, "kind": "exists" }, { "__type": "Text", "__tag": 4046, "value": ". NumPy has the machinery to do this, and it is this machinery that makes subclassing slightly non-standard." } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "There are two aspects to the machinery that ndarray uses to support views and new-from-template in subclasses." } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "The first is the use of the " }, { "__type": "InlineCode", "__tag": 4051, "value": "ndarray.__new__" }, { "__type": "Text", "__tag": 4046, "value": " method for the main work of object initialization, rather then the more usual " }, { "__type": "InlineCode", "__tag": 4051, "value": "__init__" }, { "__type": "Text", "__tag": 4046, "value": " method. The second is the use of the " }, { "__type": "InlineCode", "__tag": 4051, "value": "__array_finalize__" }, { "__type": "Text", "__tag": 4046, "value": " method to allow subclasses to clean up after the creation of views and new instances from templates." } ] } ], "title": [ { "__type": "Text", "__tag": 4046, "value": "Implications for subclassing" } ], "level": 1, "target": null }, { "__type": "Section", "__tag": 4015, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "InlineCode", "__tag": 4051, "value": "__new__" }, { "__type": "Text", "__tag": 4046, "value": " is a standard Python method, and, if present, is called before " }, { "__type": "InlineCode", "__tag": 4051, "value": "__init__" }, { "__type": "Text", "__tag": 4046, "value": " when we create a class instance. See the " }, { "__type": "Link", "__tag": 4049, "children": [ { "__type": "Text", "__tag": 4046, "value": "python __new__ documentation" } ], "url": "https://docs.python.org/reference/datamodel.html#object.__new__", "title": "" }, { "__type": "Text", "__tag": 4046, "value": " for more detail." } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "For example, consider the following Python code:" } ] }, { "__type": "Code", "__tag": 4050, "value": ">>> class C:\n... def __new__(cls, *args):\n... print('Cls in __new__:', cls)\n... print('Args in __new__:', args)\n... # The `object` type __new__ method takes a single argument.\n... return object.__new__(cls)\n... def __init__(self, *args):\n... print('type(self) in __init__:', type(self))\n... print('Args in __init__:', args)", "execution_status": null }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "meaning that we get:" } ] }, { "__type": "Code", "__tag": 4050, "value": ">>> c = C('hello')\nCls in __new__: \nArgs in __new__: ('hello',)\ntype(self) in __init__: \nArgs in __init__: ('hello',)", "execution_status": null }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "When we call " }, { "__type": "InlineCode", "__tag": 4051, "value": "C('hello')" }, { "__type": "Text", "__tag": 4046, "value": ", the " }, { "__type": "InlineCode", "__tag": 4051, "value": "__new__" }, { "__type": "Text", "__tag": 4046, "value": " method gets its own class as first argument, and the passed argument, which is the string " }, { "__type": "InlineCode", "__tag": 4051, "value": "'hello'" }, { "__type": "Text", "__tag": 4046, "value": ". After python calls " }, { "__type": "InlineCode", "__tag": 4051, "value": "__new__" }, { "__type": "Text", "__tag": 4046, "value": ", it usually (see below) calls our " }, { "__type": "InlineCode", "__tag": 4051, "value": "__init__" }, { "__type": "Text", "__tag": 4046, "value": " method, with the output of " }, { "__type": "InlineCode", "__tag": 4051, "value": "__new__" }, { "__type": "Text", "__tag": 4046, "value": " as the first argument (now a class instance), and the passed arguments following." } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "As you can see, the object can be initialized in the " }, { "__type": "InlineCode", "__tag": 4051, "value": "__new__" }, { "__type": "Text", "__tag": 4046, "value": " method or the " }, { "__type": "InlineCode", "__tag": 4051, "value": "__init__" }, { "__type": "Text", "__tag": 4046, "value": " method, or both, and in fact ndarray does not have an " }, { "__type": "InlineCode", "__tag": 4051, "value": "__init__" }, { "__type": "Text", "__tag": 4046, "value": " method, because all the initialization is done in the " }, { "__type": "InlineCode", "__tag": 4051, "value": "__new__" }, { "__type": "Text", "__tag": 4046, "value": " method." } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Why use " }, { "__type": "InlineCode", "__tag": 4051, "value": "__new__" }, { "__type": "Text", "__tag": 4046, "value": " rather than just the usual " }, { "__type": "InlineCode", "__tag": 4051, "value": "__init__" }, { "__type": "Text", "__tag": 4046, "value": "? Because in some cases, as for ndarray, we want to be able to return an object of some other class. Consider the following:" } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "meaning that:" } ] }, { "__type": "Code", "__tag": 4050, "value": ">>> obj = D('hello')\nD cls is: \nD args in __new__: ('hello',)\nCls in __new__: \nArgs in __new__: ('hello',)\n>>> type(obj)\n", "execution_status": null }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "The definition of " }, { "__type": "InlineCode", "__tag": 4051, "value": "C" }, { "__type": "Text", "__tag": 4046, "value": " is the same as before, but for " }, { "__type": "InlineCode", "__tag": 4051, "value": "D" }, { "__type": "Text", "__tag": 4046, "value": ", the " }, { "__type": "InlineCode", "__tag": 4051, "value": "__new__" }, { "__type": "Text", "__tag": 4046, "value": " method returns an instance of class " }, { "__type": "InlineCode", "__tag": 4051, "value": "C" }, { "__type": "Text", "__tag": 4046, "value": " rather than " }, { "__type": "InlineCode", "__tag": 4051, "value": "D" }, { "__type": "Text", "__tag": 4046, "value": ". Note that the " }, { "__type": "InlineCode", "__tag": 4051, "value": "__init__" }, { "__type": "Text", "__tag": 4046, "value": " method of " }, { "__type": "InlineCode", "__tag": 4051, "value": "D" }, { "__type": "Text", "__tag": 4046, "value": " does not get called. In general, when the " }, { "__type": "InlineCode", "__tag": 4051, "value": "__new__" }, { "__type": "Text", "__tag": 4046, "value": " method returns an object of class other than the class in which it is defined, the " }, { "__type": "InlineCode", "__tag": 4051, "value": "__init__" }, { "__type": "Text", "__tag": 4046, "value": " method of that class is not called." } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "This is how subclasses of the ndarray class are able to return views that preserve the class type. When taking a view, the standard ndarray machinery creates the new ndarray object with something like " } ] }, { "__type": "Code", "__tag": 4050, "value": "obj = ndarray.__new__(subtype, shape, ...", "execution_status": null }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "where " }, { "__type": "InlineCode", "__tag": 4051, "value": "subtype" }, { "__type": "Text", "__tag": 4046, "value": " is the subclass. Thus the returned view is of the same class as the subclass, rather than being of class " }, { "__type": "InlineCode", "__tag": 4051, "value": "ndarray" }, { "__type": "Text", "__tag": 4046, "value": "." } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "That solves the problem of returning views of the same type, but now we have a new problem. The machinery of ndarray can set the class this way, in its standard methods for taking views, but the ndarray " }, { "__type": "InlineCode", "__tag": 4051, "value": "__new__" }, { "__type": "Text", "__tag": 4046, "value": " method knows nothing of what we have done in our own " }, { "__type": "InlineCode", "__tag": 4051, "value": "__new__" }, { "__type": "Text", "__tag": 4046, "value": " method in order to set attributes, and so on. (Aside - why not call " }, { "__type": "InlineCode", "__tag": 4051, "value": "obj = subdtype.__new__(..." }, { "__type": "Text", "__tag": 4046, "value": " then? Because we may not have a " }, { "__type": "InlineCode", "__tag": 4051, "value": "__new__" }, { "__type": "Text", "__tag": 4046, "value": " method with the same call signature)." } ] } ], "title": [ { "__type": "Text", "__tag": 4046, "value": "A brief Python primer on " }, { "__type": "InlineCode", "__tag": 4051, "value": "__new__" }, { "__type": "Text", "__tag": 4046, "value": " and " }, { "__type": "InlineCode", "__tag": 4051, "value": "__init__" } ], "level": 2, "target": null }, { "__type": "Section", "__tag": 4015, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "InlineCode", "__tag": 4051, "value": "__array_finalize__" }, { "__type": "Text", "__tag": 4046, "value": " is the mechanism that numpy provides to allow subclasses to handle the various ways that new instances get created." } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Remember that subclass instances can come about in these three ways:" } ] }, { "__type": "BulletList", "__tag": 4053, "ordered": true, "start": 1, "children": [ { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "explicit constructor call (" }, { "__type": "InlineCode", "__tag": 4051, "value": "obj = MySubClass(params)" }, { "__type": "Text", "__tag": 4046, "value": "). This will call the usual sequence of " }, { "__type": "InlineCode", "__tag": 4051, "value": "MySubClass.__new__" }, { "__type": "Text", "__tag": 4046, "value": " then (if it exists) " }, { "__type": "InlineCode", "__tag": 4051, "value": "MySubClass.__init__" }, { "__type": "Text", "__tag": 4046, "value": "." } ] } ] }, { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "CrossRef", "__tag": 4002, "value": "view-casting", "reference": { "__type": "LocalRef", "__tag": 4022, "kind": "docs", "path": "user:basics.subclassing" }, "kind": "exists" } ] } ] }, { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "CrossRef", "__tag": 4002, "value": "new-from-template", "reference": { "__type": "LocalRef", "__tag": 4022, "kind": "docs", "path": "user:basics.subclassing" }, "kind": "exists" } ] } ] } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Our " }, { "__type": "InlineCode", "__tag": 4051, "value": "MySubClass.__new__" }, { "__type": "Text", "__tag": 4046, "value": " method only gets called in the case of the explicit constructor call, so we can't rely on " }, { "__type": "InlineCode", "__tag": 4051, "value": "MySubClass.__new__" }, { "__type": "Text", "__tag": 4046, "value": " or " }, { "__type": "InlineCode", "__tag": 4051, "value": "MySubClass.__init__" }, { "__type": "Text", "__tag": 4046, "value": " to deal with the view casting and new-from-template. It turns out that " }, { "__type": "InlineCode", "__tag": 4051, "value": "MySubClass.__array_finalize__" }, { "__type": "Text", "__tag": 4046, "value": " " }, { "__type": "Emphasis", "__tag": 4047, "children": [ { "__type": "Text", "__tag": 4046, "value": "does" } ] }, { "__type": "Text", "__tag": 4046, "value": " get called for all three methods of object creation, so this is where our object creation housekeeping usually goes." } ] }, { "__type": "BulletList", "__tag": 4053, "ordered": false, "start": 1, "children": [ { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "For the explicit constructor call, our subclass will need to create a new ndarray instance of its own class. In practice this means that we, the authors of the code, will need to make a call to " }, { "__type": "InlineCode", "__tag": 4051, "value": "ndarray.__new__(MySubClass,...)" }, { "__type": "Text", "__tag": 4046, "value": ", a class-hierarchy prepared call to " }, { "__type": "InlineCode", "__tag": 4051, "value": "super().__new__(cls, ...)" }, { "__type": "Text", "__tag": 4046, "value": ", or do view casting of an existing array (see below)" } ] } ] }, { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "For view casting and new-from-template, the equivalent of " }, { "__type": "InlineCode", "__tag": 4051, "value": "ndarray.__new__(MySubClass,..." }, { "__type": "Text", "__tag": 4046, "value": " is called, at the C level." } ] } ] } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "The arguments that " }, { "__type": "InlineCode", "__tag": 4051, "value": "__array_finalize__" }, { "__type": "Text", "__tag": 4046, "value": " receives differ for the three methods of instance creation above." } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "The following code allows us to look at the call sequences and arguments:" } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Now:" } ] }, { "__type": "Code", "__tag": 4050, "value": ">>> # Explicit constructor\n>>> c = C((10,))\nIn __new__ with class \nIn array_finalize:\n self type is \n obj type is \nIn __init__ with class \n>>> # View casting\n>>> a = np.arange(10)\n>>> cast_a = a.view(C)\nIn array_finalize:\n self type is \n obj type is \n>>> # Slicing (example of new-from-template)\n>>> cv = c[:1]\nIn array_finalize:\n self type is \n obj type is ", "execution_status": null }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "The signature of " }, { "__type": "InlineCode", "__tag": 4051, "value": "__array_finalize__" }, { "__type": "Text", "__tag": 4046, "value": " is " } ] }, { "__type": "Code", "__tag": 4050, "value": "def __array_finalize__(self, obj):", "execution_status": null }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "One sees that the " }, { "__type": "InlineCode", "__tag": 4051, "value": "super" }, { "__type": "Text", "__tag": 4046, "value": " call, which goes to " }, { "__type": "InlineCode", "__tag": 4051, "value": "ndarray.__new__" }, { "__type": "Text", "__tag": 4046, "value": ", passes " }, { "__type": "InlineCode", "__tag": 4051, "value": "__array_finalize__" }, { "__type": "Text", "__tag": 4046, "value": " the new object, of our own class (" }, { "__type": "InlineCode", "__tag": 4051, "value": "self" }, { "__type": "Text", "__tag": 4046, "value": ") as well as the object from which the view has been taken (" }, { "__type": "InlineCode", "__tag": 4051, "value": "obj" }, { "__type": "Text", "__tag": 4046, "value": "). As you can see from the output above, the " }, { "__type": "InlineCode", "__tag": 4051, "value": "self" }, { "__type": "Text", "__tag": 4046, "value": " is always a newly created instance of our subclass, and the type of " }, { "__type": "InlineCode", "__tag": 4051, "value": "obj" }, { "__type": "Text", "__tag": 4046, "value": " differs for the three instance creation methods:" } ] }, { "__type": "BulletList", "__tag": 4053, "ordered": false, "start": 1, "children": [ { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "When called from the explicit constructor, " }, { "__type": "InlineCode", "__tag": 4051, "value": "obj" }, { "__type": "Text", "__tag": 4046, "value": " is " }, { "__type": "InlineCode", "__tag": 4051, "value": "None" } ] } ] }, { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "When called from view casting, " }, { "__type": "InlineCode", "__tag": 4051, "value": "obj" }, { "__type": "Text", "__tag": 4046, "value": " can be an instance of any subclass of ndarray, including our own." } ] } ] }, { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "When called in new-from-template, " }, { "__type": "InlineCode", "__tag": 4051, "value": "obj" }, { "__type": "Text", "__tag": 4046, "value": " is another instance of our own subclass, that we might use to update the new " }, { "__type": "InlineCode", "__tag": 4051, "value": "self" }, { "__type": "Text", "__tag": 4046, "value": " instance." } ] } ] } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Because " }, { "__type": "InlineCode", "__tag": 4051, "value": "__array_finalize__" }, { "__type": "Text", "__tag": 4046, "value": " is the only method that always sees new instances being created, it is the sensible place to fill in instance defaults for new object attributes, among other tasks." } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "This may be clearer with an example." } ] } ], "title": [ { "__type": "Text", "__tag": 4046, "value": "The role of " }, { "__type": "InlineCode", "__tag": 4051, "value": "__array_finalize__" } ], "level": 2, "target": null }, { "__type": "Section", "__tag": 4015, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Using the object looks like this:" } ] }, { "__type": "Blockquote", "__tag": 4059, "children": [ { "__type": "Code", "__tag": 4050, "value": ">>> obj = InfoArray(shape=(3,)) # explicit constructor\n>>> type(obj)\n\n>>> obj.info is None\nTrue\n>>> obj = InfoArray(shape=(3,), info='information')\n>>> obj.info\n'information'\n>>> v = obj[1:] # new-from-template - here - slicing\n>>> type(v)\n\n>>> v.info\n'information'\n>>> arr = np.arange(10)\n>>> cast_arr = arr.view(InfoArray) # view casting\n>>> type(cast_arr)\n\n>>> cast_arr.info is None\nTrue", "execution_status": null } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "This class isn't very useful, because it has the same constructor as the bare ndarray object, including passing in buffers and shapes and so on. We would probably prefer the constructor to be able to take an already formed ndarray from the usual numpy calls to " }, { "__type": "InlineCode", "__tag": 4051, "value": "np.array" }, { "__type": "Text", "__tag": 4046, "value": " and return an object." } ] } ], "title": [ { "__type": "Text", "__tag": 4046, "value": "Simple example - adding an extra attribute to ndarray" } ], "level": 1, "target": null }, { "__type": "Section", "__tag": 4015, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Here is a class that takes a standard ndarray that already exists, casts as our type, and adds an extra attribute." } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "So:" } ] }, { "__type": "Blockquote", "__tag": 4059, "children": [ { "__type": "Code", "__tag": 4050, "value": ">>> arr = np.arange(5)\n>>> obj = RealisticInfoArray(arr, info='information')\n>>> type(obj)\n\n>>> obj.info\n'information'\n>>> v = obj[1:]\n>>> type(v)\n\n>>> v.info\n'information'", "execution_status": null } ] } ], "title": [ { "__type": "Text", "__tag": 4046, "value": "Slightly more realistic example - attribute added to existing array" } ], "level": 1, "target": null }, { "__type": "Section", "__tag": 4015, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "A subclass can override what happens when executing numpy ufuncs on it by overriding the default " }, { "__type": "InlineCode", "__tag": 4051, "value": "ndarray.__array_ufunc__" }, { "__type": "Text", "__tag": 4046, "value": " method. This method is executed " }, { "__type": "Emphasis", "__tag": 4047, "children": [ { "__type": "Text", "__tag": 4046, "value": "instead" } ] }, { "__type": "Text", "__tag": 4046, "value": " of the ufunc and should return either the result of the operation, or " }, { "__type": "InlineRole", "__tag": 4003, "value": "NotImplemented", "domain": null, "role": "obj", "inventory": null }, { "__type": "Text", "__tag": 4046, "value": " if the operation requested is not implemented." } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "The signature of " }, { "__type": "InlineCode", "__tag": 4051, "value": "__array_ufunc__" }, { "__type": "Text", "__tag": 4046, "value": " is " } ] }, { "__type": "Code", "__tag": 4050, "value": "def __array_ufunc__(self, ufunc, method, *inputs, **kwargs):", "execution_status": null }, { "__type": "BulletList", "__tag": 4053, "ordered": false, "start": 1, "children": [ { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Emphasis", "__tag": 4047, "children": [ { "__type": "Text", "__tag": 4046, "value": "ufunc" } ] }, { "__type": "Text", "__tag": 4046, "value": " is the ufunc object that was called." } ] } ] }, { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Emphasis", "__tag": 4047, "children": [ { "__type": "Text", "__tag": 4046, "value": "method" } ] }, { "__type": "Text", "__tag": 4046, "value": " is a string indicating how the Ufunc was called, either " }, { "__type": "InlineCode", "__tag": 4051, "value": "\"__call__\"" }, { "__type": "Text", "__tag": 4046, "value": " to indicate it was called directly, or one of its " }, { "__type": "InlineRole", "__tag": 4003, "value": "methods", "domain": null, "role": "ref", "inventory": null }, { "__type": "Text", "__tag": 4046, "value": ": " }, { "__type": "InlineCode", "__tag": 4051, "value": "\"reduce\"" }, { "__type": "Text", "__tag": 4046, "value": ", " }, { "__type": "InlineCode", "__tag": 4051, "value": "\"accumulate\"" }, { "__type": "Text", "__tag": 4046, "value": ", " }, { "__type": "InlineCode", "__tag": 4051, "value": "\"reduceat\"" }, { "__type": "Text", "__tag": 4046, "value": ", " }, { "__type": "InlineCode", "__tag": 4051, "value": "\"outer\"" }, { "__type": "Text", "__tag": 4046, "value": ", or " }, { "__type": "InlineCode", "__tag": 4051, "value": "\"at\"" }, { "__type": "Text", "__tag": 4046, "value": "." } ] } ] }, { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Emphasis", "__tag": 4047, "children": [ { "__type": "Text", "__tag": 4046, "value": "inputs" } ] }, { "__type": "Text", "__tag": 4046, "value": " is a tuple of the input arguments to the " }, { "__type": "InlineCode", "__tag": 4051, "value": "ufunc" } ] } ] }, { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Emphasis", "__tag": 4047, "children": [ { "__type": "Text", "__tag": 4046, "value": "kwargs" } ] }, { "__type": "Text", "__tag": 4046, "value": " contains any optional or keyword arguments passed to the function. This includes any " }, { "__type": "InlineCode", "__tag": 4051, "value": "out" }, { "__type": "Text", "__tag": 4046, "value": " arguments, which are always contained in a tuple." } ] } ] } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "A typical implementation would convert any inputs or outputs that are instances of one's own class, pass everything on to a superclass using " }, { "__type": "InlineCode", "__tag": 4051, "value": "super()" }, { "__type": "Text", "__tag": 4046, "value": ", and finally return the results after possible back-conversion. An example, taken from the test case " }, { "__type": "InlineCode", "__tag": 4051, "value": "test_ufunc_override_with_super" }, { "__type": "Text", "__tag": 4046, "value": " in " }, { "__type": "InlineCode", "__tag": 4051, "value": "_core/tests/test_umath.py" }, { "__type": "Text", "__tag": 4046, "value": ", is the following." } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "So, this class does not actually do anything interesting: it just converts any instances of its own to regular ndarray (otherwise, we'd get infinite recursion!), and adds an " }, { "__type": "InlineCode", "__tag": 4051, "value": "info" }, { "__type": "Text", "__tag": 4046, "value": " dictionary that tells which inputs and outputs it converted. Hence, e.g.," } ] }, { "__type": "Code", "__tag": 4050, "value": ">>> a = np.arange(5.).view(A)\n>>> b = np.sin(a)\n>>> b.info\n{'inputs': [0]}\n>>> b = np.sin(np.arange(5.), out=(a,))\n>>> b.info\n{'outputs': [0]}\n>>> a = np.arange(5.).view(A)\n>>> b = np.ones(1).view(A)\n>>> c = a + b\n>>> c.info\n{'inputs': [0, 1]}\n>>> a += b\n>>> a.info\n{'inputs': [0, 1], 'outputs': [0]}", "execution_status": null }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Note that another approach would be to use " }, { "__type": "InlineCode", "__tag": 4051, "value": "getattr(ufunc, methods)(*inputs, **kwargs)" }, { "__type": "Text", "__tag": 4046, "value": " instead of the " }, { "__type": "InlineCode", "__tag": 4051, "value": "super" }, { "__type": "Text", "__tag": 4046, "value": " call. For this example, the result would be identical, but there is a difference if another operand also defines " }, { "__type": "InlineCode", "__tag": 4051, "value": "__array_ufunc__" }, { "__type": "Text", "__tag": 4046, "value": ". E.g., let's assume that we evaluate " }, { "__type": "InlineCode", "__tag": 4051, "value": "np.add(a, b)" }, { "__type": "Text", "__tag": 4046, "value": ", where " }, { "__type": "InlineCode", "__tag": 4051, "value": "b" }, { "__type": "Text", "__tag": 4046, "value": " is an instance of another class " }, { "__type": "InlineCode", "__tag": 4051, "value": "B" }, { "__type": "Text", "__tag": 4046, "value": " that has an override. If you use " }, { "__type": "InlineCode", "__tag": 4051, "value": "super" }, { "__type": "Text", "__tag": 4046, "value": " as in the example, " }, { "__type": "InlineCode", "__tag": 4051, "value": "ndarray.__array_ufunc__" }, { "__type": "Text", "__tag": 4046, "value": " will notice that " }, { "__type": "InlineCode", "__tag": 4051, "value": "b" }, { "__type": "Text", "__tag": 4046, "value": " has an override, which means it cannot evaluate the result itself. Thus, it will return " }, { "__type": "InlineRole", "__tag": 4003, "value": "NotImplemented", "domain": null, "role": null, "inventory": null }, { "__type": "Text", "__tag": 4046, "value": " and so will our class " }, { "__type": "InlineCode", "__tag": 4051, "value": "A" }, { "__type": "Text", "__tag": 4046, "value": ". Then, control will be passed over to " }, { "__type": "InlineCode", "__tag": 4051, "value": "b" }, { "__type": "Text", "__tag": 4046, "value": ", which either knows how to deal with us and produces a result, or does not and returns " }, { "__type": "InlineRole", "__tag": 4003, "value": "NotImplemented", "domain": null, "role": null, "inventory": null }, { "__type": "Text", "__tag": 4046, "value": ", raising a " }, { "__type": "InlineCode", "__tag": 4051, "value": "TypeError" }, { "__type": "Text", "__tag": 4046, "value": "." } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "If instead, we replace our " }, { "__type": "InlineCode", "__tag": 4051, "value": "super" }, { "__type": "Text", "__tag": 4046, "value": " call with " }, { "__type": "InlineCode", "__tag": 4051, "value": "getattr(ufunc, method)" }, { "__type": "Text", "__tag": 4046, "value": ", we effectively do " }, { "__type": "InlineCode", "__tag": 4051, "value": "np.add(a.view(np.ndarray), b)" }, { "__type": "Text", "__tag": 4046, "value": ". Again, " }, { "__type": "InlineCode", "__tag": 4051, "value": "B.__array_ufunc__" }, { "__type": "Text", "__tag": 4046, "value": " will be called, but now it sees an " }, { "__type": "InlineCode", "__tag": 4051, "value": "ndarray" }, { "__type": "Text", "__tag": 4046, "value": " as the other argument. Likely, it will know how to handle this, and return a new instance of the " }, { "__type": "InlineCode", "__tag": 4051, "value": "B" }, { "__type": "Text", "__tag": 4046, "value": " class to us. Our example class is not set up to handle this, but it might well be the best approach if, e.g., one were to re-implement " }, { "__type": "InlineCode", "__tag": 4051, "value": "MaskedArray" }, { "__type": "Text", "__tag": 4046, "value": " using " }, { "__type": "InlineCode", "__tag": 4051, "value": "__array_ufunc__" }, { "__type": "Text", "__tag": 4046, "value": "." } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "As a final note: if the " }, { "__type": "InlineCode", "__tag": 4051, "value": "super" }, { "__type": "Text", "__tag": 4046, "value": " route is suited to a given class, an advantage of using it is that it helps in constructing class hierarchies. E.g., suppose that our other class " }, { "__type": "InlineCode", "__tag": 4051, "value": "B" }, { "__type": "Text", "__tag": 4046, "value": " also used the " }, { "__type": "InlineCode", "__tag": 4051, "value": "super" }, { "__type": "Text", "__tag": 4046, "value": " in its " }, { "__type": "InlineCode", "__tag": 4051, "value": "__array_ufunc__" }, { "__type": "Text", "__tag": 4046, "value": " implementation, and we created a class " }, { "__type": "InlineCode", "__tag": 4051, "value": "C" }, { "__type": "Text", "__tag": 4046, "value": " that depended on both, i.e., " }, { "__type": "InlineCode", "__tag": 4051, "value": "class C(A, B)" }, { "__type": "Text", "__tag": 4046, "value": " (with, for simplicity, not another " }, { "__type": "InlineCode", "__tag": 4051, "value": "__array_ufunc__" }, { "__type": "Text", "__tag": 4046, "value": " override). Then any ufunc on an instance of " }, { "__type": "InlineCode", "__tag": 4051, "value": "C" }, { "__type": "Text", "__tag": 4046, "value": " would pass on to " }, { "__type": "InlineCode", "__tag": 4051, "value": "A.__array_ufunc__" }, { "__type": "Text", "__tag": 4046, "value": ", the " }, { "__type": "InlineCode", "__tag": 4051, "value": "super" }, { "__type": "Text", "__tag": 4046, "value": " call in " }, { "__type": "InlineCode", "__tag": 4051, "value": "A" }, { "__type": "Text", "__tag": 4046, "value": " would go to " }, { "__type": "InlineCode", "__tag": 4051, "value": "B.__array_ufunc__" }, { "__type": "Text", "__tag": 4046, "value": ", and the " }, { "__type": "InlineCode", "__tag": 4051, "value": "super" }, { "__type": "Text", "__tag": 4046, "value": " call in " }, { "__type": "InlineCode", "__tag": 4051, "value": "B" }, { "__type": "Text", "__tag": 4046, "value": " would go to " }, { "__type": "InlineCode", "__tag": 4051, "value": "ndarray.__array_ufunc__" }, { "__type": "Text", "__tag": 4046, "value": ", thus allowing " }, { "__type": "InlineCode", "__tag": 4051, "value": "A" }, { "__type": "Text", "__tag": 4046, "value": " and " }, { "__type": "InlineCode", "__tag": 4051, "value": "B" }, { "__type": "Text", "__tag": 4046, "value": " to collaborate." } ] } ], "title": [ { "__type": "InlineCode", "__tag": 4051, "value": "__array_ufunc__" }, { "__type": "Text", "__tag": 4046, "value": " for ufuncs" } ], "level": 1, "target": "array-ufunc" }, { "__type": "Section", "__tag": 4015, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Prior to numpy 1.13, the behaviour of ufuncs could only be tuned using " }, { "__type": "InlineCode", "__tag": 4051, "value": "__array_wrap__" }, { "__type": "Text", "__tag": 4046, "value": " and " }, { "__type": "InlineCode", "__tag": 4051, "value": "__array_prepare__" }, { "__type": "Text", "__tag": 4046, "value": " (the latter is now removed). These two allowed one to change the output type of a ufunc, but, in contrast to " }, { "__type": "InlineCode", "__tag": 4051, "value": "__array_ufunc__" }, { "__type": "Text", "__tag": 4046, "value": ", did not allow one to make any changes to the inputs. It is hoped to eventually deprecate these, but " }, { "__type": "InlineCode", "__tag": 4051, "value": "__array_wrap__" }, { "__type": "Text", "__tag": 4046, "value": " is also used by other numpy functions and methods, such as " }, { "__type": "InlineCode", "__tag": 4051, "value": "squeeze" }, { "__type": "Text", "__tag": 4046, "value": ", so at the present time is still needed for full functionality." } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Conceptually, " }, { "__type": "InlineCode", "__tag": 4051, "value": "__array_wrap__" }, { "__type": "Text", "__tag": 4046, "value": " \"wraps up the action\" in the sense of allowing a subclass to set the type of the return value and update attributes and metadata. Let's show how this works with an example. First we return to the simpler example subclass, but with a different name and some print statements:" } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "We run a ufunc on an instance of our new array:" } ] }, { "__type": "Code", "__tag": 4050, "value": ">>> obj = MySubClass(np.arange(5), info='spam')\nIn __array_finalize__:\n self is MySubClass([0, 1, 2, 3, 4])\n obj is array([0, 1, 2, 3, 4])\n>>> arr2 = np.arange(5)+1\n>>> ret = np.add(arr2, obj)\nIn __array_wrap__:\n self is MySubClass([0, 1, 2, 3, 4])\n arr is array([1, 3, 5, 7, 9])\nIn __array_finalize__:\n self is MySubClass([1, 3, 5, 7, 9])\n obj is MySubClass([0, 1, 2, 3, 4])\n>>> ret\nMySubClass([1, 3, 5, 7, 9])\n>>> ret.info\n'spam'", "execution_status": null }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Note that the ufunc (" }, { "__type": "InlineCode", "__tag": 4051, "value": "np.add" }, { "__type": "Text", "__tag": 4046, "value": ") has called the " }, { "__type": "InlineCode", "__tag": 4051, "value": "__array_wrap__" }, { "__type": "Text", "__tag": 4046, "value": " method with arguments " }, { "__type": "InlineCode", "__tag": 4051, "value": "self" }, { "__type": "Text", "__tag": 4046, "value": " as " }, { "__type": "InlineCode", "__tag": 4051, "value": "obj" }, { "__type": "Text", "__tag": 4046, "value": ", and " }, { "__type": "InlineCode", "__tag": 4051, "value": "out_arr" }, { "__type": "Text", "__tag": 4046, "value": " as the (ndarray) result of the addition. In turn, the default " }, { "__type": "InlineCode", "__tag": 4051, "value": "__array_wrap__" }, { "__type": "Text", "__tag": 4046, "value": " (" }, { "__type": "InlineCode", "__tag": 4051, "value": "ndarray.__array_wrap__" }, { "__type": "Text", "__tag": 4046, "value": ") has cast the result to class " }, { "__type": "InlineCode", "__tag": 4051, "value": "MySubClass" }, { "__type": "Text", "__tag": 4046, "value": ", and called " }, { "__type": "InlineCode", "__tag": 4051, "value": "__array_finalize__" }, { "__type": "Text", "__tag": 4046, "value": " - hence the copying of the " }, { "__type": "InlineCode", "__tag": 4051, "value": "info" }, { "__type": "Text", "__tag": 4046, "value": " attribute. This has all happened at the C level." } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "But, we could do anything we wanted:" } ] }, { "__type": "Code", "__tag": 4050, "value": ">>> arr1 = np.arange(5)\n>>> obj = arr1.view(SillySubClass)\n>>> arr2 = np.arange(5)\n>>> ret = np.multiply(obj, arr2)\n>>> ret\n'I lost your data'", "execution_status": null }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "So, by defining a specific " }, { "__type": "InlineCode", "__tag": 4051, "value": "__array_wrap__" }, { "__type": "Text", "__tag": 4046, "value": " method for our subclass, we can tweak the output from ufuncs. The " }, { "__type": "InlineCode", "__tag": 4051, "value": "__array_wrap__" }, { "__type": "Text", "__tag": 4046, "value": " method requires " }, { "__type": "InlineCode", "__tag": 4051, "value": "self" }, { "__type": "Text", "__tag": 4046, "value": ", then an argument - which is the result of the ufunc or another NumPy function - and an optional parameter " }, { "__type": "Emphasis", "__tag": 4047, "children": [ { "__type": "Text", "__tag": 4046, "value": "context" } ] }, { "__type": "Text", "__tag": 4046, "value": ". This parameter is passed by ufuncs as a 3-element tuple: (name of the ufunc, arguments of the ufunc, domain of the ufunc), but is not passed by other numpy functions. Though, as seen above, it is possible to do otherwise, " }, { "__type": "InlineCode", "__tag": 4051, "value": "__array_wrap__" }, { "__type": "Text", "__tag": 4046, "value": " should return an instance of its containing class. See the masked array subclass for an implementation. " }, { "__type": "InlineCode", "__tag": 4051, "value": "__array_wrap__" }, { "__type": "Text", "__tag": 4046, "value": " is always passed a NumPy array which may or may not be a subclass (usually of the caller)." } ] } ], "title": [ { "__type": "InlineCode", "__tag": 4051, "value": "__array_wrap__" }, { "__type": "Text", "__tag": 4046, "value": " for ufuncs and other functions" } ], "level": 1, "target": "array-wrap" }, { "__type": "Section", "__tag": 4015, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "One of the problems that ndarray solves is keeping track of memory ownership of ndarrays and their views. Consider the case where we have created an ndarray, " }, { "__type": "InlineCode", "__tag": 4051, "value": "arr" }, { "__type": "Text", "__tag": 4046, "value": " and have taken a slice with " }, { "__type": "InlineCode", "__tag": 4051, "value": "v = arr[1:]" }, { "__type": "Text", "__tag": 4046, "value": ". The two objects are looking at the same memory. NumPy keeps track of where the data came from for a particular array or view, with the " }, { "__type": "InlineCode", "__tag": 4051, "value": "base" }, { "__type": "Text", "__tag": 4046, "value": " attribute:" } ] }, { "__type": "Code", "__tag": 4050, "value": ">>> # A normal ndarray, that owns its own data\n>>> arr = np.zeros((4,))\n>>> # In this case, base is None\n>>> arr.base is None\nTrue\n>>> # We take a view\n>>> v1 = arr[1:]\n>>> # base now points to the array that it derived from\n>>> v1.base is arr\nTrue\n>>> # Take a view of a view\n>>> v2 = v1[1:]\n>>> # base points to the original array that it was derived from\n>>> v2.base is arr\nTrue", "execution_status": null }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "In general, if the array owns its own memory, as for " }, { "__type": "InlineCode", "__tag": 4051, "value": "arr" }, { "__type": "Text", "__tag": 4046, "value": " in this case, then " }, { "__type": "InlineCode", "__tag": 4051, "value": "arr.base" }, { "__type": "Text", "__tag": 4046, "value": " will be None - there are some exceptions to this - see the numpy book for more details." } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "The " }, { "__type": "InlineCode", "__tag": 4051, "value": "base" }, { "__type": "Text", "__tag": 4046, "value": " attribute is useful in being able to tell whether we have a view or the original array. This in turn can be useful if we need to know whether or not to do some specific cleanup when the subclassed array is deleted. For example, we may only want to do the cleanup if the original array is deleted, but not the views. For an example of how this can work, have a look at the " }, { "__type": "InlineCode", "__tag": 4051, "value": "memmap" }, { "__type": "Text", "__tag": 4046, "value": " class in " }, { "__type": "InlineCode", "__tag": 4051, "value": "numpy._core" }, { "__type": "Text", "__tag": 4046, "value": "." } ] } ], "title": [ { "__type": "Text", "__tag": 4046, "value": "Extra gotchas - custom " }, { "__type": "InlineCode", "__tag": 4051, "value": "__del__" }, { "__type": "Text", "__tag": 4046, "value": " methods and ndarray.base" } ], "level": 1, "target": null }, { "__type": "Section", "__tag": 4015, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "When sub-classing " }, { "__type": "InlineCode", "__tag": 4051, "value": "ndarray" }, { "__type": "Text", "__tag": 4046, "value": " or creating duck-types that mimic the " }, { "__type": "InlineCode", "__tag": 4051, "value": "ndarray" }, { "__type": "Text", "__tag": 4046, "value": " interface, it is your responsibility to decide how aligned your APIs will be with those of numpy. For convenience, many numpy functions that have a corresponding " }, { "__type": "InlineCode", "__tag": 4051, "value": "ndarray" }, { "__type": "Text", "__tag": 4046, "value": " method (e.g., " }, { "__type": "InlineCode", "__tag": 4051, "value": "sum" }, { "__type": "Text", "__tag": 4046, "value": ", " }, { "__type": "InlineCode", "__tag": 4051, "value": "mean" }, { "__type": "Text", "__tag": 4046, "value": ", " }, { "__type": "InlineCode", "__tag": 4051, "value": "take" }, { "__type": "Text", "__tag": 4046, "value": ", " }, { "__type": "InlineCode", "__tag": 4051, "value": "reshape" }, { "__type": "Text", "__tag": 4046, "value": ") work by checking if the first argument to a function has a method of the same name. If it exists, the method is called instead of coercing the arguments to a numpy array." } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "For example, if you want your sub-class or duck-type to be compatible with numpy's " }, { "__type": "InlineCode", "__tag": 4051, "value": "sum" }, { "__type": "Text", "__tag": 4046, "value": " function, the method signature for this object's " }, { "__type": "InlineCode", "__tag": 4051, "value": "sum" }, { "__type": "Text", "__tag": 4046, "value": " method should be the following:" } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "This is the exact same method signature for " }, { "__type": "InlineCode", "__tag": 4051, "value": "np.sum" }, { "__type": "Text", "__tag": 4046, "value": ", so now if a user calls " }, { "__type": "InlineCode", "__tag": 4051, "value": "np.sum" }, { "__type": "Text", "__tag": 4046, "value": " on this object, numpy will call the object's own " }, { "__type": "InlineCode", "__tag": 4051, "value": "sum" }, { "__type": "Text", "__tag": 4046, "value": " method and pass in these arguments enumerated above in the signature, and no errors will be raised because the signatures are completely compatible with each other." } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "If, however, you decide to deviate from this signature and do something like this:" } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "This object is no longer compatible with " }, { "__type": "InlineCode", "__tag": 4051, "value": "np.sum" }, { "__type": "Text", "__tag": 4046, "value": " because if you call " }, { "__type": "InlineCode", "__tag": 4051, "value": "np.sum" }, { "__type": "Text", "__tag": 4046, "value": ", it will pass in unexpected arguments " }, { "__type": "InlineCode", "__tag": 4051, "value": "out" }, { "__type": "Text", "__tag": 4046, "value": " and " }, { "__type": "InlineCode", "__tag": 4051, "value": "keepdims" }, { "__type": "Text", "__tag": 4046, "value": ", causing a TypeError to be raised." } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "If you wish to maintain compatibility with numpy and its subsequent versions (which might add new keyword arguments) but do not want to surface all of numpy's arguments, your function's signature should accept " }, { "__type": "InlineCode", "__tag": 4051, "value": "**kwargs" }, { "__type": "Text", "__tag": 4046, "value": ". For example:" } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "This object is now compatible with " }, { "__type": "InlineCode", "__tag": 4051, "value": "np.sum" }, { "__type": "Text", "__tag": 4046, "value": " again because any extraneous arguments (i.e. keywords that are not " }, { "__type": "InlineCode", "__tag": 4051, "value": "axis" }, { "__type": "Text", "__tag": 4046, "value": " or " }, { "__type": "InlineCode", "__tag": 4051, "value": "dtype" }, { "__type": "Text", "__tag": 4046, "value": ") will be hidden away in the " }, { "__type": "InlineCode", "__tag": 4051, "value": "**unused_kwargs" }, { "__type": "Text", "__tag": 4046, "value": " parameter." } ] } ], "title": [ { "__type": "Text", "__tag": 4046, "value": "Subclassing and downstream compatibility" } ], "level": 1, "target": null } ], "local_refs": [] }