weight initialization example

[sandhu-githhub]

Dear Community,

can someone please guide me, how to initialize weights. I could not see any example.
I found some functions in the docs for "Functional" but could not understand!!!

For example in another similar platform "DeepXDE", the network weights can be initialized as "Glorot Uniform" distribution.
I believe such initialization would help my problem convergence!!!

thanks in anticipation

[ehsanhaghighat]

Check the MLPFunctional docstring: https://github.com/sciann/sciann/blob/master/sciann/functionals/mlp_functional.py <https://github.com/sciann/sciann/blob/master/sciann/functionals/mlp_functional.py> When defining a functional, you can pass kernel_initializer or bias_initializer as arguments: e.g.

> fxy = sn.Functional(‘fxy’, [x, y], 4*[20], ’tanh’, kernel_initializer=keras.initializers.RandomNormal(stddev=0.01))

Alternatively, once you define the network using default initializer, you can get and set weights as:

> fxy.get_weights()

or

…

> fxy.set_weights( ***list of weights similar to what you get from fxy.get_weights()*** ) On Sep 5, 2022, at 12:02 AM, sandhu-githhub ***@***.***> wrote: Dear Community, can someone please guide me, how to initialize weights. I could not see any example. I found some functions in the docs for "Functional" but could not understand!!! For example in another similar platform "DeepXDE", the network weights can be initialized as "Glorot Uniform" distribution. I believe such initialization would help my problem convergence!!! thanks in anticipation — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you are subscribed to this thread.

[sandhu-githhub]

Dear @ehsanhaghighat

huge respect and thanks .....

[sandhu-githhub]

Dear @ehsanhaghighat

I would need your assistance once more...!!! Please

I just installed the latest version of sciann on a GPU machine.... everythig went fine....

Previously i used to assign multiple activation functions irrelevant of the number of layers..... It helps convergence...

Here is a snippet of my Functional definition

layers = [250]*4
z_train = Variable('z_train',dtype='float64')
x_train = Variable('x_train',dtype='float64')
m_train = Variable('m_train',dtype='float64')
U0r_train = Variable('U0r_train',dtype='float64')
U0i_train = Variable('U0i_train',dtype='float64')
dUr = Variable('dUr',dtype='float64')
dUi = Variable('dUi',dtype='float64')

dUr,dUi = Functional(['dUr','dUi'],[z_train,x_train],layers,['atan','elu','swish','sin','cos'])

"I learnt that, i cannot use l-sin and l-cos", so i used sin and cos instead....

However while running the model i am getting the error:

FailedPreconditionError: 2 root error(s) found.
(0) FAILED_PRECONDITION: Could not find variable sci_rowdy_activation_layer_1/alpha1. This could mean that the variable has been deleted. In TF1, it can also mean the variable is uninitialized. Debug info: container=localhost, status error message=Resource localhost/sci_rowdy_activation_layer_1/alpha1/N10tensorflow3VarE does not exist.
[[{{node sci_rowdy_activation_layer_1/ReadVariableOp_1}}]]
[[loss_1/Identity_1/_4247]]
(1) FAILED_PRECONDITION: Could not find variable sci_rowdy_activation_layer_1/alpha1. This could mean that the variable has been deleted. In TF1, it can also mean the variable is uninitialized. Debug info: container=localhost, status error message=Resource localhost/sci_rowdy_activation_layer_1/alpha1/N10tensorflow3VarE does not exist.
[[{{node sci_rowdy_activation_layer_1/ReadVariableOp_1}}]]
0 successful operations.
0 derived errors ignored.

Seeking your help!!!!!