Population size

[vanquanTRAN]

Look into the FAQ of the readme. Can the bug be resolved by one of those solutions?

Describe the bug

Code to reproduce the behavior

Error message from command line

System information:

• OS Platform and Distribution
• Python version
• Hyperactive version

Additional context
Dear Simon,
i have one question relating the population size of PSO,
search_space,
initialize={"grid": 4, "random": 2, "vertices": 4},
population=10,
inertia=0.5,
cognitive_weight=0.5,
social_weight=0.5,
temp_weight=0.2,
rand_rest_p=0.03,
)
i can not increase the population size which is automatically reduced to 10 when i modifies to 20 or greater ? i tried to midifiy on file search.py

if random_state is None:
    random_state = np.random.randint(0, high=2 ** 32 - 2, dtype=np.int64)

but it is not changed anything, so can you help me to debug this issue please ?
I would investigate the population effect on time-consuming and cost function of my problem.
Thank you for your help

[SimonBlanke]

Hello @vanquanTRAN,

the initial population is generated from initial positions. Increase the initial positions to a number bigger than the population, by setting: initialize={"grid": 4, "random": 15, "vertices": 4}

The current warning is:
Warning: Not enough initial positions for population size
Population size is reduced to, n_inits

I will change this to make it clear how to increase the population.

[vanquanTRAN]

I tried to modify all parameters of initialize={"grid": 100, "random": 100, "vertices": 2000}, however with population size 50, the current warning always stay
Warning: Not enough initial positions for population size
Population size is reduced to 10
How can i fix this problem ?

[SimonBlanke]

Hello @vanquanTRAN,

I need a complete example that I can copy and run. Additionally I need the information about the python and hyperactive version + your OS.

[vanquanTRAN]

-- coding: utf-8 --

"""
Created on Sun Jun 6 13:06:22 2021

@author: Admin
"""
data_tv.csv
import numpy as np
import pandas as pd

import matplotlib.pyplot as plt
plt.rcParams["font.family"] = "Times New Roman"
plt.rcParams.update({'font.size': 22})
import matplotlib
matplotlib.rc('xtick', labelsize=14)
matplotlib.rc('ytick', labelsize=14)
matplotlib.pyplot.figure(dpi=1200)
import seaborn as sns
from sklearn.metrics import mean_squared_error, mean_absolute_error, r2_score
from sklearn.preprocessing import StandardScaler
from sklearn.model_selection import train_test_split
import xgboost as xgb
from sklearn.ensemble import RandomForestRegressor, AdaBoostRegressor, GradientBoostingRegressor
from sklearn.svm import SVR
from numpy.random import seed
seed(2)

data = pd.read_csv(r"data_tv.csv")

#sns.pairplot(data)
#Splitting train and test data
X = data.iloc[:,:-1] # Features
y = data.iloc[:,-1] # Target
sc = StandardScaler()
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3)
X_train = sc.fit_transform(X_train)
X_test = sc.transform(X_test)

from sklearn.model_selection import cross_val_score
from hyperactive import Hyperactive, RandomRestartHillClimbingOptimizer,BayesianOptimizer,ParticleSwarmOptimizer,SimulatedAnnealingOptimizer,RandomAnnealingOptimizer

def model(opt):
gbr = RandomForestRegressor(n_estimators=opt["n_estimators"],

                            max_depth=opt["max_depth"],
                            max_features=opt["max_features"],
                            min_samples_split=opt["min_samples_split"],
                            min_samples_leaf=opt["min_samples_leaf"])

scores=np.mean(cross_val_score(gbr, X_train, y_train,cv=10,  n_jobs=-1,
                                scoring="r2"))
print(
    "Iteration:", opt.optimizer.nth_iter, " Best score", opt.optimizer.best_score
)
return 1*scores

search_space = {"n_estimators": list(range(500, 600,10)),

            "max_depth": list(range(4, 9,2)),
            "max_features": list(range(4, 6,1)),
            "min_samples_split": list(np.arange(0.001, 0.05,0.001)),
            "min_samples_leaf": list(np.arange(0.001, 0.05,0.001))}

optimizer = ParticleSwarmOptimizer(population=50,
inertia=0.4,
cognitive_weight=0.7,
social_weight=0.7,
temp_weight=0.3,
rand_rest_p=0.05,
)

hyper = Hyperactive()
hyper.add_search(model, search_space, optimizer=optimizer, n_iter=500)
hyper.run()

[vanquanTRAN]

i use hyperactive 3.2.3

[SimonBlanke]

@vanquanTRAN,

in your example code I do not see how you used the initialize-parameter to increase the number of initial positions.

[vanquanTRAN]

import numpy as np

from .base_population_optimizer import BasePopulationOptimizer
from ...search import Search
from ._particle import Particle

class ParticleSwarmOptimizer(BasePopulationOptimizer, Search):
def init(
self,
search_space,
initialize={"grid": 100, "random": 100, "vertices": 2000},
population=10,
inertia=0.5,
cognitive_weight=0.5,
social_weight=0.5,
temp_weight=0.2,
rand_rest_p=0.03,
):
super().init(search_space, initialize)

    self.population = population
    self.inertia = inertia
    self.cognitive_weight = cognitive_weight
    self.social_weight = social_weight
    self.temp_weight = temp_weight
    self.rand_rest_p = rand_rest_p

    self.particles = self._create_population(Particle)
    self.optimizers = self.particles

def _sort_best(self):
    scores_list = []
    for _p_ in self.particles:
        scores_list.append(_p_.score_current)

    scores_np = np.array(scores_list)
    idx_sorted_ind = list(scores_np.argsort()[::-1])

    self.p_sorted = [self.particles[i] for i in idx_sorted_ind]

def init_pos(self, pos):
    nth_pop = self.nth_iter % len(self.particles)

    self.p_current = self.particles[nth_pop]
    self.p_current.init_pos(pos)

    self.p_current.inertia = self.inertia
    self.p_current.cognitive_weight = self.cognitive_weight
    self.p_current.social_weight = self.social_weight
    self.p_current.temp_weight = self.temp_weight
    self.p_current.rand_rest_p = self.rand_rest_p

    self.p_current.velo = np.zeros(len(self.conv.max_positions))

def iterate(self):
    n_iter = self._iterations(self.particles)
    self.p_current = self.particles[n_iter % len(self.particles)]

    self._sort_best()
    self.p_current.global_pos_best = self.p_sorted[0].pos_best

    pos = self.p_current.iterate()

    return pos

def evaluate(self, score_new):
    self.p_current.evaluate(score_new)

I modified in the particle_swarm_optimization.py of package gradient_free_optimizers

[SimonBlanke]

Hello @vanquanTRAN,

it looks like you are changing the source code of Hyperactive. You should not do this. The way Hyperactive behaves should only be changed by using the official API:
https://github.com/SimonBlanke/Hyperactive#hyperactive-api-reference

If you want to learn how to use Hyperactive you can also look into the tutorial:
https://github.com/SimonBlanke/hyperactive-tutorial/blob/main/notebooks/hyperactive_tutorial.ipynb