first- and second-order derivatives

Question

first- and second-order derivatives

weiweiWYW opened this issue 4 years ago · 4 comments

In your paper, only first-order derivatives are used, but in LightGBM, second order derivatives are usually used as well. Does DPBoost only rely on first-order derivatives?

Answer 1 · 2021-03-24T08:24:20.000Z

Hi @weiweiWYW ,

DPBoost considers second-order derivative as 1. Note that in LightGBM the second-order derivative is a constant (i.e., 1) if adopting squared loss.

Answer 2 · 2021-03-24T10:07:47.000Z

Hi @QinbinLi ,

I'm mainly dealing with binary classification. In DPBoost, it is treated as regression, and then output is converted to a binary value. But if I use binary cross entropy, does mean that I could just use one-order derivative to compute the G and V ? which are shown in your paper.

Answer 3 · 2021-03-24T14:41:05.000Z

Hi @weiweiWYW ,

I think only using one-order derivative is an alternative approach. However, its accuracy may not be as good as using both one-order and second-order derivatives.

Answer 4 · 2021-03-25T05:49:53.000Z

OK, thank you for your reply