Request for SQDMetal + Palace Example Files
Closed this issue · 19 comments
Hi all,
First off, great work on the project! I was wondering if you could upload an example file that demonstrates how to use SQDMetal to design and run simulations in Palace on a local PC?
Specifically, I’m looking for examples like running an eigenmodal simulation of a half-wave resonator or calculating the capacitance matrix of a floating transmon qubit using Palace. Or really just any complete design to simulation flow....
Some more complete tutorial/example files would be really appreciated! 🙏🏽
Thanks for all your hard work.
Best,
Shanto
I was following the running eigenmodal sim documentation that y'all have using the following toy code - A CPW resonator connected to a port.
import numpy as np
from collections import OrderedDict
from qiskit_metal import designs, draw
from qiskit_metal import MetalGUI, Dict, Headings
from qiskit_metal.qlibrary.tlines.meandered import RouteMeander
from qiskit_metal.qlibrary.tlines.straight_path import RouteStraight
from qiskit_metal.qlibrary.terminations.launchpad_wb import LaunchpadWirebond
from qiskit_metal.qlibrary.terminations.launchpad_wb_coupled import LaunchpadWirebondCoupled
from qiskit_metal.qlibrary.terminations.short_to_ground import ShortToGround
from qiskit_metal.qlibrary.terminations.open_to_ground import OpenToGround
short_to_ground = False
print(design.variables['cpw_width'])
print(design.variables['cpw_gap'])
design._chips['main']['size']['size_x'] = '8mm'
design._chips['main']['size']['size_y'] = '5mm'
# we also set metal thickness to 0
print(design.ls.ls_df)
design.ls.ls_df.loc[0, 'thickness'] = '0 um' # top PEC layer has no thickess (2D sim)
print(design.ls.ls_df)
# instantiate wirebond pad
wb_fd_1 = LaunchpadWirebond(design, 'wb_fd_1', options = dict(pos_x='-2mm', pos_y='0mm', orientation='0', layer=1, chip="main"))
# instantiate OTGs instead, a little smaller
otg_2 = OpenToGround(design, 'otg-2', options = dict(pos_x = '2mm', orientation='0'))
# instantiate meander resonator
meander_opts = dict(
pin_inputs = dict(
start_pin = dict(component = "wb_fd_1", pin = "tie"),
# end_pin = dict(component = "wb_fd_2", pin = "tie")
#start_pin = dict(component = "otg-1", pin = "open"),
end_pin = dict(component = "otg-2", pin = "open")
),
lead = dict(
start_straight = "500um",
),
total_length = '30mm',
fillet = "99.99um",
asymmetry = "100um",
chip="main",
layer=1
)
resonator = RouteMeander(design, 'res_1', meander_opts)
# check to see the build
gui.rebuild()
gui.autoscale()
# connect to SQDMetal
from SQDMetal.PALACE.Eigenmode_Simulation import PALACE_Eigenmode_Simulation
from SQDMetal.PALACE.SQDGmshRenderer import Palace_Gmsh_Renderer
#Eigenmode Simulation Options
user_defined_options = {
"mesh_refinement": 0, #refines mesh in PALACE - essetially divides every mesh element in half
"dielectric_material": "silicon", #choose dielectric material - 'silicon' or 'sapphire'
"starting_freq": 7.5, #starting frequency in GHz
"number_of_freqs": 4, #number of eigenmodes to find
"solns_to_save": 4, #number of electromagnetic field visualizations to save
"solver_order": 2, #increasing solver order increases accuracy of simulation, but significantly increases sim time
"solver_tol": 1.0e-8, #error residual tolerance foriterative solver
"solver_maxits": 100, #number of solver iterations
"comsol_meshing": "Extremely fine", #level of COMSOL meshing: 'Extremely fine', 'Extra fine', 'Finer', 'Fine', 'Normal'
"mesh_max": 120e-3, #maxiumum element size for the mesh in mm
"mesh_min": 10e-3, #minimum element size for the mesh in mm
"mesh_sampling": 130, #number of points to mesh along a geometry
"sim_memory": '300G', #amount of memory for each HPC node i.e. 4 nodes x 300 GB = 1.2 TB
"sim_time": '20:00:00', #allocated time for simulation
"HPC_nodes": '4', #number of Bunya nodes. By default 20 cpus per node are selected, then total cores = 20 x HPC_nodes
"fillet_resolution":12 #Number of vertices per quarter turn on a filleted path
}
#Creat the Palace Eigenmode simulation
eigen_sim = PALACE_Eigenmode_Simulation(name ='single_resonator_example_eigen', #name of simulation
metal_design = design, #feed in qiskit metal design
sim_parent_directory = "test", #choose directory where mesh file, config file and HPC batch file will be saved
mode = 'simPC', #choose simulation mode 'HPC' or 'simPC'
meshing = 'GMSH', #choose meshing 'GMSH' or 'COMSOL'
user_options = user_defined_options, #provide options chosen above
view_design_gmsh_gui = False, #view design in GMSH gui
create_files = True)
eigen_sim.add_metallic(1)
eigen_sim.add_ground_plane()
eigen_sim.create_port_CPW_on_Launcher('wb_fd_1', 20e-3)
eigen_sim.fine_mesh_along_path(100e-6, 'res_1', mesh_sampling=130, mesh_min=5e-3, mesh_max=120e-3)
eigen_sim.prepare_simulation()I get the following error at the eigen_sim.prepare_simulation() command.
{
"name": "NameError",
"message": "name 'options' is not defined",
"stack": "---------------------------------------------------------------------------
NameError Traceback (most recent call last)
Cell In[17], line 1
----> 1 eigen_sim.prepare_simulation()
File ~/LFL/SQDMetal/SQDMetal/PALACE/Model.py:36, in PALACE_Model.prepare_simulation(self)
35 def prepare_simulation(self):
---> 36 self._prepare_simulation(self._metallic_layers, self._ground_plane)
File ~/LFL/SQDMetal/SQDMetal/PALACE/Model.py:249, in PALACE_Model_RF_Base._prepare_simulation(self, metallic_layers, ground_plane)
246 lePorts += [(cur_port['port_name'] + 'b', cur_port['portBcoords'])]
248 #prepare design by converting shapely geometries to Gmsh geometries
--> 249 gmsh_render_attrs = pgr._prepare_design(metallic_layers, ground_plane, lePorts, options['fillet_resolution'], 'eigenmode_simulation')
251 if self.create_files == True:
252 #create directory to store simulation files
253 self._create_directory(self.name)
NameError: name 'options' is not defined"
}I am not sure what is causing this and how I can fix this.... I do have a working executable of palace and I am sure I have to let SQDMetal know about this but I am not sure where to pass on that information....
Hello,
Firstly, thank you for your interest in SQDMetal! Anyway, the errors you saw were due to bugs that have been fixed by the latest push/commit. Here is the snippets from the ipynb that makes it work without throwing errors:
import numpy as np
from collections import OrderedDict
import qiskit_metal as metal
from qiskit_metal import designs, draw
from qiskit_metal import MetalGUI, Dict, Headings
from qiskit_metal.qlibrary.tlines.meandered import RouteMeander
from qiskit_metal.qlibrary.tlines.straight_path import RouteStraight
from qiskit_metal.qlibrary.terminations.launchpad_wb import LaunchpadWirebond
from qiskit_metal.qlibrary.terminations.launchpad_wb_coupled import LaunchpadWirebondCoupled
from qiskit_metal.qlibrary.terminations.short_to_ground import ShortToGround
from qiskit_metal.qlibrary.terminations.open_to_ground import OpenToGround# Create a DesignPlanar (class to handle 2D designs)
design = metal.designs.DesignPlanar(metadata = Dict(design_name='CPW-Test'))
design.overwrite_enabled = True # Enables overwriting componentsshort_to_ground = False
print(design.variables['cpw_width'])
print(design.variables['cpw_gap'])
design._chips['main']['size']['size_x'] = '8mm'
design._chips['main']['size']['size_y'] = '5mm'
# we also set metal thickness to 0
# print(design.ls.ls_df)
# design.ls.ls_df.loc[0, 'thickness'] = '0 um' # top PEC layer has no thickess (2D sim)
# print(design.ls.ls_df)
# instantiate wirebond pad
wb_fd_1 = LaunchpadWirebond(design, 'wb_fd_1', options = dict(pos_x='-2mm', pos_y='0mm', orientation='0', layer=1, chip="main"))
# instantiate OTGs instead, a little smaller
otg_2 = OpenToGround(design, 'otg-2', options = dict(pos_x = '2mm', orientation='0'))
# instantiate meander resonator
meander_opts = dict(
pin_inputs = dict(
start_pin = dict(component = "wb_fd_1", pin = "tie"),
# end_pin = dict(component = "wb_fd_2", pin = "tie")
#start_pin = dict(component = "otg-1", pin = "open"),
end_pin = dict(component = "otg-2", pin = "open")
),
lead = dict(
start_straight = "500um",
),
total_length = '30mm',
fillet = "99.99um",
asymmetry = "100um",
chip="main",
layer=1
)
resonator = RouteMeander(design, 'res_1', meander_opts)# gui = MetalGUI(design)
# gui.rebuild()
# gui.autoscale()# connect to SQDMetal
from SQDMetal.PALACE.Eigenmode_Simulation import PALACE_Eigenmode_Simulation
from SQDMetal.PALACE.SQDGmshRenderer import Palace_Gmsh_Renderer
#Eigenmode Simulation Options
user_defined_options = {
"mesh_refinement": 0, #refines mesh in PALACE - essetially divides every mesh element in half
"dielectric_material": "silicon", #choose dielectric material - 'silicon' or 'sapphire'
"starting_freq": 7.5, #starting frequency in GHz
"number_of_freqs": 4, #number of eigenmodes to find
"solns_to_save": 4, #number of electromagnetic field visualizations to save
"solver_order": 2, #increasing solver order increases accuracy of simulation, but significantly increases sim time
"solver_tol": 1.0e-8, #error residual tolerance foriterative solver
"solver_maxits": 100, #number of solver iterations
"comsol_meshing": "Extremely fine", #level of COMSOL meshing: 'Extremely fine', 'Extra fine', 'Finer', 'Fine', 'Normal'
"mesh_max": 120e-3, #maxiumum element size for the mesh in mm
"mesh_min": 10e-3, #minimum element size for the mesh in mm
"mesh_sampling": 130, #number of points to mesh along a geometry
"sim_memory": '300G', #amount of memory for each HPC node i.e. 4 nodes x 300 GB = 1.2 TB
"sim_time": '20:00:00', #allocated time for simulation
"HPC_nodes": '4', #number of Bunya nodes. By default 20 cpus per node are selected, then total cores = 20 x HPC_nodes
"fillet_resolution":12, #Number of vertices per quarter turn on a filleted path
"palace_dir":"/path-to-palace-in-spack/bin/palace"
}
#Creat the Palace Eigenmode simulation
eigen_sim = PALACE_Eigenmode_Simulation(name ='single_resonator_example_eigen', #name of simulation
metal_design = design, #feed in qiskit metal design
sim_parent_directory = "test/", #choose directory where mesh file, config file and HPC batch file will be saved
mode = 'simPC', #choose simulation mode 'HPC' or 'simPC'
meshing = 'GMSH', #choose meshing 'GMSH' or 'COMSOL'
user_options = user_defined_options, #provide options chosen above
view_design_gmsh_gui = False, #view design in GMSH gui
create_files = True)
eigen_sim.add_metallic(1)
eigen_sim.add_ground_plane()
eigen_sim.create_port_CPW_on_Launcher('wb_fd_1', 20e-3)
eigen_sim.fine_mesh_along_path(20e-6, 'res_1', mesh_sampling=130, mesh_min=5e-3, mesh_max=120e-3)
eigen_sim.prepare_simulation()Make sure to replace the path in the above snippet to palace...
eigen_sim.set_freq_search(2e9, 2)
data = eigen_sim.run()However, note the following:
- The number of mesh elements creates a 1GB mesh file - this crashes Palace if the workstation does not have adequate RAM (even 96GB was not enough here)
- We have a stable version when using COMSOL to perform the meshing. However,
gmshis still experimental, but we are close to certifying it into a production-ready state.
@DavidSomm Perhaps we can finalise the issues with gmsh and use the example above to demonstrate it in a working state this week?
Hi @PP501
This is great! Thank you so much for the extremely quick fix and response. I got the code to generate the .msh and .json files but it throws an errror when it tries to run palace
Error: Palace binary "/Users/shanto/LFL/palace/build/bin/palace-arm64.bin" does not match host architecture.
{
"name": "FileNotFoundError",
"message": "[Errno 2] No such file or directory: '/Users/shanto/LFL/SQDMetal/test2/single_resonator_example_eigen/outputFiles/eig.csv'",
"stack": "---------------------------------------------------------------------------
FileNotFoundError Traceback (most recent call last)
Cell In[7], line 1
----> 1 data = eigen_sim.run()
File ~/LFL/SQDMetal/SQDMetal/PALACE/Model.py:119, in PALACE_Model.run(self)
116 self.cur_process.kill()
117 self.cur_process = None
--> 119 return self.retrieve_data()
File ~/LFL/SQDMetal/SQDMetal/PALACE/Eigenmode_Simulation.py:209, in PALACE_Eigenmode_Simulation.retrieve_data(self)
208 def retrieve_data(self):
--> 209 raw_data = pd.read_csv(self._output_data_dir + '/eig.csv')
210 headers = raw_data.columns
211 raw_data = raw_data.to_numpy()
File ~/miniconda3/envs/qiskit_metal/lib/python3.10/site-packages/pandas/util/_decorators.py:211, in deprecate_kwarg.<locals>._deprecate_kwarg.<locals>.wrapper(*args, **kwargs)
209 else:
210 kwargs[new_arg_name] = new_arg_value
--> 211 return func(*args, **kwargs)
File ~/miniconda3/envs/qiskit_metal/lib/python3.10/site-packages/pandas/util/_decorators.py:331, in deprecate_nonkeyword_arguments.<locals>.decorate.<locals>.wrapper(*args, **kwargs)
325 if len(args) > num_allow_args:
326 warnings.warn(
327 msg.format(arguments=_format_argument_list(allow_args)),
328 FutureWarning,
329 stacklevel=find_stack_level(),
330 )
--> 331 return func(*args, **kwargs)
File ~/miniconda3/envs/qiskit_metal/lib/python3.10/site-packages/pandas/io/parsers/readers.py:950, in read_csv(filepath_or_buffer, sep, delimiter, header, names, index_col, usecols, squeeze, prefix, mangle_dupe_cols, dtype, engine, converters, true_values, false_values, skipinitialspace, skiprows, skipfooter, nrows, na_values, keep_default_na, na_filter, verbose, skip_blank_lines, parse_dates, infer_datetime_format, keep_date_col, date_parser, dayfirst, cache_dates, iterator, chunksize, compression, thousands, decimal, lineterminator, quotechar, quoting, doublequote, escapechar, comment, encoding, encoding_errors, dialect, error_bad_lines, warn_bad_lines, on_bad_lines, delim_whitespace, low_memory, memory_map, float_precision, storage_options)
935 kwds_defaults = _refine_defaults_read(
936 dialect,
937 delimiter,
(...)
946 defaults={\"delimiter\": \",\"},
947 )
948 kwds.update(kwds_defaults)
--> 950 return _read(filepath_or_buffer, kwds)
File ~/miniconda3/envs/qiskit_metal/lib/python3.10/site-packages/pandas/io/parsers/readers.py:605, in _read(filepath_or_buffer, kwds)
602 _validate_names(kwds.get(\"names\", None))
604 # Create the parser.
--> 605 parser = TextFileReader(filepath_or_buffer, **kwds)
607 if chunksize or iterator:
608 return parser
File ~/miniconda3/envs/qiskit_metal/lib/python3.10/site-packages/pandas/io/parsers/readers.py:1442, in TextFileReader.__init__(self, f, engine, **kwds)
1439 self.options[\"has_index_names\"] = kwds[\"has_index_names\"]
1441 self.handles: IOHandles | None = None
-> 1442 self._engine = self._make_engine(f, self.engine)
File ~/miniconda3/envs/qiskit_metal/lib/python3.10/site-packages/pandas/io/parsers/readers.py:1735, in TextFileReader._make_engine(self, f, engine)
1733 if \"b\" not in mode:
1734 mode += \"b\"
-> 1735 self.handles = get_handle(
1736 f,
1737 mode,
1738 encoding=self.options.get(\"encoding\", None),
1739 compression=self.options.get(\"compression\", None),
1740 memory_map=self.options.get(\"memory_map\", False),
1741 is_text=is_text,
1742 errors=self.options.get(\"encoding_errors\", \"strict\"),
1743 storage_options=self.options.get(\"storage_options\", None),
1744 )
1745 assert self.handles is not None
1746 f = self.handles.handle
File ~/miniconda3/envs/qiskit_metal/lib/python3.10/site-packages/pandas/io/common.py:856, in get_handle(path_or_buf, mode, encoding, compression, memory_map, is_text, errors, storage_options)
851 elif isinstance(handle, str):
852 # Check whether the filename is to be opened in binary mode.
853 # Binary mode does not support 'encoding' and 'newline'.
854 if ioargs.encoding and \"b\" not in ioargs.mode:
855 # Encoding
--> 856 handle = open(
857 handle,
858 ioargs.mode,
859 encoding=ioargs.encoding,
860 errors=errors,
861 newline=\"\",
862 )
863 else:
864 # Binary mode
865 handle = open(handle, ioargs.mode)
FileNotFoundError: [Errno 2] No such file or directory: '/Users/shanto/LFL/SQDMetal/test2/single_resonator_example_eigen/outputFiles/eig.csv'"
}
I can use those same config and mesh files to run palace with the same executable via the command line and by running ./temp.sh.
Oh I also needed to add
os.chmod("temp.sh", 0o755)
in the run method to deal with some permission issues.
I was curious if this is a SQDMetal issue of me-being-dumb issue 😅?
Again, really appreciate y'all.
Thanks for your reply (and no it is not your fault; it's more a Linux/SQDMetal issue). So a couple things:
- I have run this on 2 PCs (Kubuntu 22 and 24) and never had this permission issue
- What file-system are you using?
- I suspect that it may be to do with the execution permissions on your filesystem in general (e.g. I run these on admin accounts in general).
- Anyway, I will add the
os.chmodcommand in the next commit. But perhaps if you give the parent directory a+755, it may fix it?
- I don't think it actually ran your simulation. Can you check if you have a file
outputFiles/out.log? This should contain the raw shell output from Palace.
In the mean time, @DavidSomm and I will work to get this in a more stable condition this week; of course, we will include a minimum working example to benchmark/test on your end.
I am using a Macbook Pro (M1 Pro) where I installed palace using this guide
23.6.0 Darwin Kernel Version 23.6.0: Mon Jul 29 21:14:30 PDT 2024; root:xnu-10063.141.2~1/RELEASE_ARM64_T6000 arm64
Changing the parent directory permission to 755 did not work sadly.
You are correct - it did not run the sim because of the Error: Palace binary "/Users/shanto/LFL/palace/build/bin/palace-arm64.bin" does not match host architecture error so that is why I got the error with eig.csv not found (it was never generated).
Also, I made changes to my local repo for SQDMetal where I can pass num_cpu to be included in the temp.sh via user_defined_options - may be y'all can also include this in the future commits?
Don't worry too much about this if it is not an issue y'all didn't face before it might be some sort of Mac OS only permissions issue and its also not that big of a deal since I can always just run the shell script via the command line (the crucial part of generating the config and mesh file works so I am very happy and grateful for that :))
I figured out the issue!
My jupyter environment where I was running the code was running under x86_64 and my palace build is for arm64!
import platform
import subprocess
import os
# helper method
def is_native_arm64():
try:
# Run the sysctl command to check the native architecture
result = subprocess.run(["sysctl", "-n", "machdep.cpu.brand_string"], capture_output=True, text=True, check=True)
return "M" in result.stdout # M is for Apple M1/M2 chips
except subprocess.CalledProcessError as e:
print(f"Error checking native architecture: {e}")
return False
def run(self):
assert self._sim_config != "", "Must run prepare_simulation at least once."
config_file = self._sim_config
leFile = os.path.basename(os.path.realpath(config_file))
leDir = os.path.dirname(os.path.realpath(config_file))
if not os.path.exists(self._output_data_dir):
os.makedirs(self._output_data_dir)
log_location = f"{self._output_data_dir}/out.log"
# Write the temp.sh script
with open("temp.sh", "w+") as f:
f.write(f"cd \"{leDir}\"\n")
f.write(f"\"{self.palace_dir}\" -np {self.num_cpu} {leFile} | tee \"{log_location}\"\n")
# Set execute permission on temp.sh
print("Setting execute permission on temp.sh")
os.chmod("temp.sh", 0o755)
# Create or clear the log file
with open(log_location, 'w') as fp:
pass
# Get the current running architecture
running_arch = platform.machine()
# If the machine is native arm64 but running under x86_64, run temp.sh under arm64
if is_native_arm64() and running_arch == "x86_64":
print("Running arm64 process from x86_64 environment...")
self.cur_process = subprocess.Popen(["arch", "-arm64", "bash", "./temp.sh"], shell=False)
else:
# Run the temp.sh script as usual
print("Running temp.sh script under current architecture...")
self.cur_process = subprocess.Popen("./temp.sh", shell=True)
try:
self.cur_process.wait()
except KeyboardInterrupt:
self.cur_process.kill()
finally:
self.cur_process = None
return self.retrieve_data()People like me who use qiskit-metal on M1/M2 Macs must use Rosetta to emulate x86_64 because PySide2 lacks arm64 support (and IBM is yet to PR the PySide6 compatible version 🙄). This "fix" detects such a scenario and resolves the palace compatibility issue I faced....
Since now I have a sort of idea of how to run eigenmodal sims using palace via SQDMetal I think a tutorial for getting capacitance matrix for a TransmonCross might be super valuable 🙏🏽
Hi all,
I was wondering what the timeline is for putting together the full comprehensive example?
I will be giving a lecture for the Qiskit Fall Fest 2024 and wanted to give y'all a shoutout + show how it is possible to use palace via SQDMetal to do eigenmode simulations for CPWs (which I now know how to - thanks!) and doing an electrostatic simulation on TransmonCross qubit with a claw (one that couples it to the CPW) to get the capacitance matrix (this is the part I do not yet know how to do)
Hi @shanto268,
I am currently working on the comprehensive example and trying to get something ready. What date do you have to present at the Qiskit Fall Fest?
I have all the simulations (eigenmode, driven, capacitance) working with a COMSOL generated mesh but need to improve the accuracy with gmsh meshing. For the eigenmode simulations that you ran did you manage to get accurate resonant frequencies for your CPWs?
Cheers,
David
Hi @DavidSomm ,
My lecture is scheduled to be on the 16th (11 AM EST).
For the eigenmode sims, I did get reasonable frequencies with finer meshing but kappa was not that great. For my talk, I just wanted to show that one can run simulations on designs rendered in qiskit-metal using palace via SQDMetal API (so the accuracy for this talk is not that relevant...).
I can mention that gmsh meshing needs to be worked on more and ask the community to contribute?
Please let me know what you think.
Hello,
We would certainly appreciate the shout-out! I think that the API won't change upon fixing it. If you got the simulation working, you can check the E-field plots that are automatically generated. If they also look reasonable (i.e. it locks onto the actual standing wave modes), then it should be instructive for a lecture. We are currently just testing the meshing and simulation accuracy. @DavidSomm got stable simulations working when setting the solder to order 1. As this is not ideal (i.e. it will require unnecessarily fine meshes), we are working to alleviate that by meshing the z-axis more intelligently.
Again, apologies for the delays, but we will ensure that API will remain consistent after our upcoming fixes. Nonetheless, we will provide online examples. If you want to be specific, I suggest that you tell them that:
- Palace with COMSOL meshes enables fast local and HPC/cluster simulations
- Palace with GMSH currently requires too fine a mesh; this issue is being fixed and should be production-ready soon
Regards,
Sounds great @PP501
Please let me know once an example code for doing capacitance matrix extraction from a TransmonCross with a claw using palace is available 😇
Really appreciate the great work, y'all do!
Best,
Shanto
Hi @shanto268,
Sorry for coming back to you a little late. @PP501 and I have been working the past few days to get PALACE working nicely with gmsh. We have solved the issue which was giving us inaccurate results for higher solver orders (solver order > 2). Essentially, we were running an older version of PALACE (v0.11) that had a bug which was making it difficult to improve the accuracy of simulations. We have now updated the example for eigenmode simulations that was previously posted on the SQDMetal GitHub page, you can find it under worked examples!
We've also been working on the code for DC capacitance simulations and believe it should be working now, however I still need to test it by running a few simulations. I will try and produce a worked example (X-mon with claw) in the next couple of hours. If this is too late for your presentation, I apologise in advance. I would have liked to have sent a worked example to you earlier but we had a few bugs in the capacitance simulation code that we really needed to iron out. In any case, the eigenmode simulations should be accurate and we'll be putting more examples up shortly.
UPDATE: I've posted a worked example for a capacitance simulation for X-mon and claw here.
Hi @DavidSomm ,
Thank y'all so much for putting so much effort into this. I saw the example files and it looks great! I could not run them or include them in my "example workflow" but I am absolutely adding the link to the WorkedExamples and then post my talk (and before uploading my .ipynb file) I will incorporate these changes.
Really appreciate y'all. Do y'all plan on making a pypi release for SQDMetal?
Hi all,
I tested the eigenmode sim to work. There was one error for a key word (`leSlice does not have "E_real" for something") search but I fixed it by:
def retrieve_data(self):
raw_data = pd.read_csv(self._output_data_dir + '/eig.csv')
headers = raw_data.columns
raw_data = raw_data.to_numpy()
lePlots = self._output_data_dir + '/paraview/eigenmode/eigenmode.pvd'
if os.path.exists(lePlots):
leView = PVDVTU_Viewer(lePlots)
for m in range(leView.num_datasets):
try:
leSlice = leView.get_data_slice(m)
fig = leSlice.plot(np.linalg.norm(leSlice.get_data('E_real'), axis=1), 'coolwarm', True)
fig.savefig(self._output_data_dir + f'/eig{m}_ErealMag.png')
plt.close(fig)
except Exception as e:
print(f"Error in plotting: {e}")
try:
fig = leSlice.plot_mesh()
fig.savefig(self._output_data_dir + '/mesh.png')
plt.close(fig)
except Exception as e:
print(f"Error in plotting: {e}")There seems to be some issues with the cap matrix still I think. It did not have the self._output_data_dir defined but I addressed in a hacky way as seen below (more methods were added to work on my Apple Silicon mac but that should not cause any issue in any other OS really)
def run(self):
assert self._sim_config != "", "Must run prepare_simulation at least once."
config_file = self._sim_config
leFile = os.path.basename(os.path.realpath(config_file))
leDir = os.path.dirname(os.path.realpath(config_file))
# check if self._output_data_dir is defined
if not hasattr(self, "_output_data_dir"):
self._output_dir = "outputFiles"
print(self._output_dir)
self._output_data_dir = os.path.dirname(os.path.realpath(self._sim_config)) + "/" + self._output_dir
if not os.path.exists(self._output_data_dir):
os.makedirs(self._output_data_dir)
log_location = f"{self._output_data_dir}/out.log"
with open("temp.sh", "w+") as f:
f.write(f"cd \"{leDir}\"\n")
f.write(f"\"{self.palace_dir}\" -np {self.num_cpu} {leFile} | tee \"{log_location}\"\n")
# Set execute permission on temp.sh
print("Setting execute permission on temp.sh")
os.chmod("temp.sh", 0o755)
with open(log_location, 'w') as fp:
pass
# Get the current running architecture
running_arch = platform.machine()
# If the machine is native arm64 but running under x86_64, run temp.sh under arm64
if is_native_arm64() and running_arch == "x86_64":
print("Running arm64 process from x86_64 environment...")
self.cur_process = subprocess.Popen(["arch", "-arm64", "bash", "./temp.sh"], shell=False)
else:
# Run the temp.sh script as usual
print("Running temp.sh script under current architecture...")
self.cur_process = subprocess.Popen("./temp.sh", shell=True)
try:
self.cur_process.wait()
except KeyboardInterrupt:
self.cur_process.kill()
self.cur_process = None
return self.retrieve_data()Now that I could get it to run. I get the following error.
outputFiles
Setting execute permission on temp.sh
Running arm64 process from x86_64 environment...
>> /opt/homebrew/bin/mpirun -n 2 /Users/shanto/LFL/palace/build/bin/palace-arm64.bin xmon_cap_sim_GMSH.json
_____________ _______
_____ __ \____ __ /____ ____________
____ /_/ / __ ` / / __ ` / ___/ _ \
___ _____/ /_/ / / /_/ / /__/ ___/
/__/ \___,__/__/\___,__/\_____\_____/
Verification failed: (postpro->empty()) is false:
--> Found an unsupported configuration file keyword under "Postprocessing"!
{
"Capacitance": [
{
"Attributes": [
1
],
"Index": 1
},
{
"Attributes": [
2
],
"Index": 2
},
{
"Attributes": [
3
],
"Index": 3
},
{
"Attributes": [
4
],
"Index": 4
},
{
"Attributes": [
5
],
"Index": 5
}
]
}
... in function: void palace::config::BoundaryPostData::SetUp(json &)
... in file: /Users/shanto/LFL/palace/palace/utils/configfile.cpp:1367
Verification failed: (postpro->empty()) is false:
--> Found an unsupported configuration file keyword under "Postprocessing"!
{
"Capacitance": [
{
"Attributes": [
1
],
"Index": 1
},
{
"Attributes": [
2
],
"Index": 2
},
{
"Attributes": [
3
],
"Index": 3
},
{
"Attributes": [
4
],
"Index": 4
},
{
"Attributes": [
5
],
"Index": 5
}
]
}
... in function: void palace::config::BoundaryPostData::SetUp(json &)
... in file: /Users/shanto/LFL/palace/palace/utils/configfile.cpp:1367
--------------------------------------------------------------------------
MPI_ABORT was invoked on rank 0 in communicator MPI_COMM_WORLD
Proc: [[18805,1],0]
Errorcode: 1
NOTE: invoking MPI_ABORT causes Open MPI to kill all MPI processes.
You may or may not see output from other processes, depending on
exactly when Open MPI kills them.
--------------------------------------------------------------------------
--------------------------------------------------------------------------
prterun has exited due to process rank 0 with PID 0 on node GlobalMachine-2 calling
"abort". This may have caused other processes in the application to be
terminated by signals sent by prterun (as reported here).
--------------------------------------------------------------------------It is probably because I am running palace v0.13.0? Do y'all recommend v0.12.0 instead?
When I change the `"Boundaries" to
"Boundaries": {
"Ground": {
"Attributes": [
8
]
},
"Terminal": [
{
"Index": 1,
"Attributes": [
1
]
},
{
"Index": 2,
"Attributes": [
2
]
},
{
"Index": 3,
"Attributes": [
3
]
},
{
"Index": 4,
"Attributes": [
4
]
},
{
"Index": 5,
"Attributes": [
5
]
}
],
"Postprocessing": {
"SurfaceFlux": [
{
"Index": 1,
"Attributes": [
1
],
"Type": "Electric"
},
{
"Index": 2,
"Attributes": [
2
],
"Type": "Electric"
},
{
"Index": 3,
"Attributes": [
3
],
"Type": "Electric"
},
{
"Index": 4,
"Attributes": [
4
],
"Type": "Electric"
},
{
"Index": 5,
"Attributes": [
5
],
"Type": "Electric"
}
]
}
},
The simulation does run.
Hello,
Once again thank you for linking our repository. We may make it a PyPI project later. We just like this workflow of being able to edit the active installation. Nonetheless, if it reaches a stable state, we should do this for convenience.
As for the boundaries, for capacitance simulations, you do usually want the background to be grounded as the zero-potential reference (in COMSOL, one uses the infinite-element domain to push it towards infinity as well). This is an oversight carried over from the RF simulations I believe; we will fix that in the upcoming commit. As for the remaining errors, we will patch those errors today. Thank you for helping us test/debug this independently.
Awesome! THANK YOU guys!
Hello,
We have patched all the issues you had and updated the examples. Indeed it was due to Palace v0.13. The configuration file will not match the new specification. We will run detailed tests once we figure out how to deploy 0.13 on Ubuntu.
@PP501 Thanks Prassana.
I am closing this issue now. Please feel free to email me (shanto@usc.edu) if I can be of any assistance.
Best,
Shanto