Supra Adjacency Matrix Error
ashuein opened this issue · 40 comments
I am trying to follow the example "example_multiplex_dynamics.py". In fact, any multiplex network holds the following error.
When I try to get the supra adjacency matrix with command
multilayer_network.get_supra_adjacency_matrix()
Then I get the following error:
**File "/usr/local/lib/python3.6/dist-packages/scipy/sparse/sputils.py", line 52, in upcast
raise TypeError('no supported conversion for types: %r' % (args,))
TypeError: no supported conversion for types: (dtype('<U3'),)**
What I am not doing correctly?
Hello Ashuein, could you please try with the new version of the library?
I can not reproduce the bug, I've added the supra matrix example here for your convenience:
https://github.com/SkBlaz/Py3plex/blob/master/examples/example_supra_adjacency.py (works fine with me -> you get a sparse 500x500 matrix)
If this example works fine, please provide an example of the input data you use and how you load it, so we can debug further.
I am closing this, as it seems this is related to your Scipy installation.
Hi SkBlaz
Sorry for the late reply as I thought there won't be any response.
- I upgraded py3plex installation using pip install --upgrade py3plex
- I am running the following lines of code:
from py3plex.core import multinet
comNet = multinet.multi_layer_network().load_network('/path_to_network_file/',directed=False,input_type='multiplex_edges')
comNet.load_layer_name_mapping('/path_to_layer_map_file/')
my input files are in this format:
Input network file:
L1 nodeA nodeB 1
L1 nodeA nodeC 1
L3 nodeA nodeD 1
and so on
Layer map file:
LayerID LayerName
L1 hub
L2 transport
L3 utility
1 is the connection strength for all edges.
When I run the command:
mat = comNet.get_supra_adjacency_matrix()
I get the following error:
`Traceback (most recent call last):
File "", line 1, in
mat = comNet.get_supra_adjacency_matrix()
File "/usr/local/lib/python3.6/dist-packages/py3plex/core/multinet.py", line 629, in get_supra_adjacency_matrix
return nx.to_scipy_sparse_matrix(self.core_network)
File "/usr/local/lib/python3.6/dist-packages/networkx/convert_matrix.py", line 794, in to_scipy_sparse_matrix
for u, v, d in selfloops
File "/usr/local/lib/python3.6/dist-packages/networkx/convert_matrix.py", line 795, in
if u in index and v in index))
TypeError: bad operand type for unary -: 'str'`
I see, i did not realize you were using the multiplex parser. I'll look into that today, this is possibly a bug indeed. Thanks!
I want to add one observation. If you are using networkx.to_scipy_sparse_matrix for the multigraph then it actually doesn't return a supra adjacency matrix. The edges if exist in all layers will be aggregated by it's weight. So I don't know how will it give you supra adjacency matrix.
Hey Ashuein! I've corrected this problem, it seems there was some incorrect type casting. Please, do check if this works now: https://github.com/SkBlaz/Py3plex/blob/master/examples/example_supra_adjacency.py
I suppose I need to re-install the py3plex package. Which I did.
Running the same code again gives the following error:
Traceback (most recent call last):
File "<ipython-input-19-aa961994eae7>", line 14, in <module>
mat = comNet.get_supra_adjacency_matrix()
File "/usr/local/lib/python3.6/dist-packages/py3plex/core/multinet.py", line 629, in get_supra_adjacency_matrix
return nx.to_scipy_sparse_matrix(self.core_network)
File "/usr/local/lib/python3.6/dist-packages/networkx/convert_matrix.py", line 801, in to_scipy_sparse_matrix
return M.asformat(format)
File "/usr/local/lib/python3.6/dist-packages/scipy/sparse/base.py", line 329, in asformat return convert_method()
File "/usr/local/lib/python3.6/dist-packages/scipy/sparse/coo.py", line 400, in tocsr data = np.empty_like(self.data, dtype=upcast(self.dtype))
File "/usr/local/lib/python3.6/dist-packages/scipy/sparse/sputils.py", line 52, in upcast raise TypeError('no supported conversion for types: %r' % (args,))
TypeError: no supported conversion for types: (dtype('<U1'),)
Do I need to recompile the package manually by downloading it, or re-installing with pip will work?
Hi SkBlaz, the error is gone. Thanks for the help.
However, can you tell me why the number of nodes is different for every layer in the final network?
I ensured in my data that the nodes remain the same across all layers.
Your artificial data does contain a different set of nodes in every layer. So, I tried to run your code by modifying the input edge data. All interactions are same but existing in all the 3 layers
L1 nodeA nodeB 1
L1 nodeA nodeC 1
L1 nodeA nodeD 1
L2 nodeA nodeB 1
L2 nodeA nodeC 1
L2 nodeA nodeD 1
L3 nodeA nodeB 1
L3 nodeA nodeC 1
L3 nodeA nodeD 1
Running the code:
comNet = multinet.multi_layer_network().load_network('../Py3plex_master/datasets/simple_multiplex.edgelist',directed=False,input_type='multiplex_edges')
comNet.basic_stats()
comNet.load_layer_name_mapping('../Py3plex_master/datasets/simple_multiplex.txt')
mat = comNet.get_supra_adjacency_matrix()
Gives matrix 'mat'
<9x12 sparse matrix of type '<class 'numpy.float64'>'
with 9 stored elements in Compressed Sparse Row format>
matrix([[0., 1., 1., 1., 0., 0., 0., 0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0., 1., 1., 1., 0., 0., 0., 0.],
[0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0., 0., 0., 0., 0., 1., 1., 1.]])
Which should not happen. I am not able to understand what this matrix represents for?
A supra adjacency matrix for a multiplex network should have intralayer edge adjacency matrix along diagonals and identity matrices as an off-diagonal matrix, denoting the same number of nodes across all layers.
I suspect that the function used from networkx package to get the adjacency matrix is the flaw as the definition of multigraph it is using is not the same as the multiplex graph. Using hstack and vstack functions from numpy or scipy might help to create supra adjacency. I was able to create one, but have the problem in the plotting that matrix.
Thanks for the testing. Currently, node-layer pairs correspond to nodes in the supra-adjacency matrix, and this is implemented native scipy coo matrices: see lines 638 onward: https://github.com/SkBlaz/Py3plex/blob/master/py3plex/core/multinet.py Could you provide and example how you implemented the supra adjacency in this case? Thanks!
…
On Tue, Mar 19, 2019 at 6:37 AM Ashutosh @.***> wrote: Your artificial data does contain a different set of nodes in every layer. So, I tried to run your code by modifying the input edge data. All interactions are same but existing in all the 3 layers L1 nodeA nodeB 1 L1 nodeA nodeC 1 L1 nodeA nodeD 1 L2 nodeA nodeB 1 L2 nodeA nodeC 1 L2 nodeA nodeD 1 L3 nodeA nodeB 1 L3 nodeA nodeC 1 L3 nodeA nodeD 1 Running the code: comNet = multinet.multi_layer_network().load_network('../Py3plex_master/datasets/simple_multiplex.edgelist',directed=False,input_type='multiplex_edges') comNet.basic_stats() comNet.load_layer_name_mapping('../Py3plex_master/datasets/simple_multiplex.txt') mat = comNet.get_supra_adjacency_matrix() Gives matrix 'mat' <9x12 sparse matrix of type '<class 'numpy.float64'>' with 9 stored elements in Compressed Sparse Row format> matrix([[0., 1., 1., 1., 0., 0., 0., 0., 0., 0., 0., 0.], [0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.], [0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.], [0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.], [0., 0., 0., 0., 0., 1., 1., 1., 0., 0., 0., 0.], [0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.], [0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.], [0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.], [0., 0., 0., 0., 0., 0., 0., 0., 0., 1., 1., 1.]]) Which should not happen. I am not able to understand what this matrix represents for? A supra adjacency matrix for a multiplex network should have intralayer edge adjacency matrix along diagonals and identity matrices as an off-diagonal matrix, denoting the same number of nodes across all layers. I suspect that the function used from networkx package to get the adjacency matrix is the flaw as the definition of multigraph it is using is not the same as the multiplex graph. Using hstack and vstack functions from numpy or scipy might help to create supra adjacency. I was able to create so but have the problem in the plot the same matrix. — You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub <#3 (comment)>, or mute the thread https://github.com/notifications/unsubscribe-auth/AJkiRC9uL_F-I-zR0gjSILPlmac58s3Bks5vYHeigaJpZM4bDXo4 .
I had only three layers to make so used hstack and vstack functions
adj_L1, adj_L2 and adj_L3 are adjacency matrices of each layer
common_nodes = [A,B,C,D]
crossLay = np.identity(len(common_nodes))
matA = np.hstack((adj_L1,crossLay,crossLay))
matB = np.hstack((crossLay,adj_L2,crossLay))
matC = np.hstack((crossLay,crossLay,adj_L3))
supra_adj_matrix = np.vstack((matA,matB,matC))
For very large number of layers, I would use loop to achieve the above stackings
Assume number of layers to be M,
crossLay = np.identity(len(common_nodes))
adj = [adj_L1,adj_L2,adj_L3,....,adj_M]
var = [ ]
for i in range(M):
tmp = crossLay*M ; tmp[i] = adj[i]; tmp = np.matrix(tmp)
var.append(tmp); tmp = [ ]
var = np.vstack(tuple(var))
See the updated example, it returns 9x9 matrix as it should now.
…
On Tue, Mar 19, 2019 at 6:47 AM Blaž Škrlj @.> wrote: Thanks for the testing. Currently, node-layer pairs correspond to nodes in the supra-adjacency matrix, and this is implemented native scipy coo matrices: see lines 638 onward: https://github.com/SkBlaz/Py3plex/blob/master/py3plex/core/multinet.py Could you provide and example how you implemented the supra adjacency in this case? Thanks! On Tue, Mar 19, 2019 at 6:37 AM Ashutosh @.> wrote: > Your artificial data does contain a different set of nodes in every > layer. So, I tried to run your code by modifying the input edge data. All > interactions are same but existing in all the 3 layers > > L1 nodeA nodeB 1 > L1 nodeA nodeC 1 > L1 nodeA nodeD 1 > L2 nodeA nodeB 1 > L2 nodeA nodeC 1 > L2 nodeA nodeD 1 > L3 nodeA nodeB 1 > L3 nodeA nodeC 1 > L3 nodeA nodeD 1 > > Running the code: > > comNet = > multinet.multi_layer_network().load_network('../Py3plex_master/datasets/simple_multiplex.edgelist',directed=False,input_type='multiplex_edges') > comNet.basic_stats() > > comNet.load_layer_name_mapping('../Py3plex_master/datasets/simple_multiplex.txt') > mat = comNet.get_supra_adjacency_matrix() > > Gives matrix 'mat' > > <9x12 sparse matrix of type '<class 'numpy.float64'>' > with 9 stored elements in Compressed Sparse Row format> > > matrix([[0., 1., 1., 1., 0., 0., 0., 0., 0., 0., 0., 0.], > [0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.], > [0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.], > [0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.], > [0., 0., 0., 0., 0., 1., 1., 1., 0., 0., 0., 0.], > [0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.], > [0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.], > [0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.], > [0., 0., 0., 0., 0., 0., 0., 0., 0., 1., 1., 1.]]) > > Which should not happen. I am not able to understand what this matrix > represents for? > A supra adjacency matrix for a multiplex network should have intralayer > edge adjacency matrix along diagonals and identity matrices as an > off-diagonal matrix, denoting the same number of nodes across all layers. > I suspect that the function used from networkx package to get the > adjacency matrix is the flaw as the definition of multigraph it is using is > not the same as the multiplex graph. Using hstack and vstack functions from > numpy or scipy might help to create supra adjacency. I was able to create > so but have the problem in the plot the same matrix. > > — > You are receiving this because you modified the open/close state. > Reply to this email directly, view it on GitHub > <#3 (comment)>, or mute > the thread > https://github.com/notifications/unsubscribe-auth/AJkiRC9uL_F-I-zR0gjSILPlmac58s3Bks5vYHeigaJpZM4bDXo4 > . >
Which example you are mentioning?
So, I tried after re-installing your py3plex package and seems that the supra adjacency matrix is now symmetric. But still, the non-diagonal matrices of supra matrix is a null matrix, rather than being an identity matrix. Here is the result when applied on ../datasets/simple_multiplex.edgelist
matrix([[0., 1., 1., 1., 0., 0., 0., 0., 0., 0., 0., 0.],
[1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
[1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
[1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0., 1., 1., 1., 0., 0., 0., 0.],
[0., 0., 0., 0., 1., 0., 0., 0., 0., 0., 0., 0.],
[0., 0., 0., 0., 1., 0., 0., 0., 0., 0., 0., 0.],
[0., 0., 0., 0., 1., 0., 0., 0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0., 0., 0., 0., 0., 1., 1., 1.],
[0., 0., 0., 0., 0., 0., 0., 0., 1., 0., 0., 0.],
[0., 0., 0., 0., 0., 0., 0., 0., 1., 0., 0., 0.],
[0., 0., 0., 0., 0., 0., 0., 0., 1., 0., 0., 0.]])
Which is a 12 by 12 matrix as it should be, but off-diagonal matrices should be an identity for a multiplex network denoting coupling across layers. The issue of ordering of node can be fixed by keeping a dictionary of node index and node name after the names have been sorted internally.
Also, yes, it would be great to contribute to the method as I proposed.
If the initial py3plex object is specified with network_type = "multiplex", the couplings are added after the file is read, and this should work just fine. Could you please verify that this is the case?
I can, if you would like, add you as a contributor, if you see this can be better formulated.
the code which gets the matrix is located from lines 638 onwards in py3plex/core/multinet.py file.
Please see the supra adjacencey matrix example: https://github.com/SkBlaz/Py3plex/blob/master/examples/example_supra_adjacency.py
You can now obtain the row names easily.
I can, if you would like, add you as a contributor, if you see this can be better formulated.
the code which gets the matrix is located from lines 638 onwards in py3plex/core/multinet.py file.
I will look into the multinet.py file. But can you be little more specific on what better formulation you are proposing? So that I get an idea.
Please see the supra adjacencey matrix example: https://github.com/SkBlaz/Py3plex/blob/master/examples/example_supra_adjacency.py
You can now obtain the row names easily.
I tried this, it does give the node order but the issue is node numbers are continuous across layers. In a multilayer network, this is Ok because every layer does not have the same nodes. But for a multiplex network, by definition, every layer has the same set of nodes. It's only the edge structure that changes. Hence, with this result, it is impossible to know which node we are talking about except the first layer. Besides, it also doesn't clarify which node number belongs to which layer.
Apart from the above two, I found one more thing to be considered. In a multiplex network, the number of nodes remains the same in all the layers, but every node may not be connected in all layers. Nodes which are not connected with an edge in a layer (node degree is 0 for that layer), can be called as orphan nodes. But because it is connected in other layers, it is important to hold it in every layer. I found that your implementation doesn't consider this point and simply consolidates all layer nodes ( which is also called layer aggregation). This violates the very definition of a multiplex network.
This problem is arising because your input file format enforces a two node interaction format and doesn't compare with the node index file. If you take all the three files as in your example, the edges, the nodes and the layers, then you can easily first add edges for every layer in networkx (those nodes will be created automatically). The compare the nodes in all the layers individually with the node id file and add orphan nodes, if any, to the layer where it is not present. This way you preserve the node numbers for every layer and networkx then can generate an appropriate matrix for you. Which can be used to create supra matrix.
I would also suggest to only ask for only node names (both node input file and edge table) and create a dictionary inside the code. This way you can keep the nodes sorted and hence the indexing will be in order in the output.
Node names can also be derived directly from edge tables, but the layer-wise node number correction is a must.
This doesn't seem to be true. Node names consist of (node,layer) tuples, hence this should be tractable, or am I missing something? To be more specific, if you look at the: def _encode_to_numeric(self): method, there could be another similar method, which returns a supra matrix for a multiplex graph. This is where you could implement the way you construct the multiplex supra matrix, if that makes sense?
…
On Fri, Mar 22, 2019 at 7:37 AM Ashutosh @.***> wrote: Please see the supra adjacencey matrix example: https://github.com/SkBlaz/Py3plex/blob/master/examples/example_supra_adjacency.py You can now obtain the row names easily. I tried this, it does give the node order but the issue is node numbers are continuous across layers. In a multilayer network, this is Ok because every layer does not have the same nodes. But for a multiplex network, by definition, every layer has the same set of nodes. It's only the edge structure that changes. Hence, with this result, it is impossible to know which node we are talking about except the first layer. Besides, it also doesn't clarify which node number belongs to which layer. — You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub <#3 (comment)>, or mute the thread https://github.com/notifications/unsubscribe-auth/AJkiRKPC7Yjsv5AEbxcBY3hDniMOqu50ks5vZHpGgaJpZM4bDXo4 .
When I run this command on the example:
comNet.node_order_in_matrix(data=True)
I get the following output:
NodeDataView({0: {}, 1: {}, 2: {}, 3: {}, 4: {}, 5: {}, 6: {}, 7: {}, 8: {}, 9: {}, 10: {}, 11: {}})
If I remove data=True then
NodeView((0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11))
Am I getting this right?
Thanks for this suggestion! Indeed, our implementation considers a network multiplex, if the same node appears in at least one other layer, hence by your definition, this is not ok. I'll have time next week to add this layer-wise correction, yet if you would like to take a stab at this before, I'd be glad to include your solution. Maybe even faster way would be to simply add nodes for each layer first, and then introduce edges. This is very useful feedback :)
…
On Fri, Mar 22, 2019 at 7:50 AM Ashutosh @.***> wrote: Apart from the above two, I found one more thing to be considered. In a multiplex network, the number of nodes remains the same in all the layers, but every node may not be connected in all layers. Nodes which are not connected with an edge in a layer (node degree is 0 for that layer), can be called as orphan nodes. But because it is connected in other layers, it is important to hold it in every layer. I found that your implementation doesn't consider this point and simply consolidates all layer nodes ( which is also called layer aggregation). This violates the very definition of a multiplex network. This problem is arising because your input file format enforces a two node interaction format and doesn't compare with the node index file. If you take all the three files as in your example, the edges, the nodes and the layers, then you can easily first add edges for every layer in networkx (those nodes will be created automatically). The compare the nodes in all the layers individually with the node id file and add orphan nodes, if any, to the layer where it is not present. This way you preserve the node numbers for every layer and networkx then can generate an appropriate matrix for you. Which can be used to create supra matrix. I would also suggest to only ask for only node names (both node input file and edge table) and create a dictionary inside the code. This way you can keep the nodes sorted and hence the indexing will be in order in the output. Node names can also be derived directly from edge tables, but the layer-wise node number correction is a must. — You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub <#3 (comment)>, or mute the thread https://github.com/notifications/unsubscribe-auth/AJkiREC5x-LK7mnIffx7WkaNrsj8wR7fks5vZH0XgaJpZM4bDXo4 .
can you tell me where exactly in the package code I can modify to add nodes? I will at least do it for my purposes.
I tried to modify the function 'parse_multiplex_edges' in 'parsers.py' file to include those nodes which are not present in any layer. I could do it successfully. But on running the example, the problem comes in the function 'add_mpx_edges' which is applicable if the network_type=multiplex parameter is passed on. I am not able to understand this 'add_mpx_edges' function defined in supporting.py file. So, I think to bring the above changes is not straight forward, at least for me.
In your above example, example_manipulation.py, I am not able to understand where do you define layer while adding the node? Hence I went for the above-said modification, which of course didn't work as well.
Hello Ashuein! I've rewritten this part of the code, please check the supra adjacency example - this now contains all information (along with couplings). Is this what you had in mind?
The supra adjacency matrix is still not creating the off Diagonal matrices as identity matrix.
If you refer to https://arxiv.org/abs/1407.0742 page 7 and 17, you will get an idea what I am talking about.
I've added this version of supra matrix (see image below). It is now given as the supra example.
The implementation is now in accordance with the formulation in the paper. Please inform for any further issues.