Generation failure, TypeError: ones(): argument 'size' must be tuple of ints, but found element of type Tensor at pos 1

[pranavpawar3]

Hi Team,

I am able to install and import the SEAL modules. Also able to create the index, and load it with the checkpoint.

However, when I try to perform any search, it gives the following error -
I reckon it might be occurring due to torch and transformer version incompatibility. I am using torch - 1.11.0 and transformers - 4.13.0

I am planning to debug the beam_search script if required, please let me know if I shall try something else instead.

Thanks,
Pranav

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-14-722411077b17> in <module>
----> 1 searcher.search("how to find my account number", k=3)

projects/seal-exp/SEAL/seal/retrieval.py in search(self, query, k, added_documents, detokenize)
    645         if added_documents is not None:
    646             added_documents = [added_documents]
--> 647         return self.batch_search([query], k=k, added_documents=added_documents, detokenize=True)[0]
    648 
    649     def batch_search(self, queries, k: int = 10, added_documents=None, detokenize=None) -> List[List[SEALDocument]]:

projects/seal-exp/SEAL/seal/retrieval.py in batch_search(self, queries, k, added_documents, detokenize)
    658                 keys = ((kk, None, added_documents[i]) for i, kk in enumerate(keys))
    659 
--> 660         results, keys = zip(*self.batch_retrieve_from_keys(keys))
    661 
    662         keys = list({k for kk in keys for k in kk})

projects/seal-exp/SEAL/seal/retrieval.py in batch_retrieve_from_keys(self, keys)
    758             yield from self._mp_batch_retrieve_from_keys(keys)
    759         else:
--> 760             yield from self._batch_retrieve_from_keys(keys)
    761 
    762     def _mp_batch_retrieve_from_keys(self, keys):

projects/seal-exp/SEAL/seal/retrieval.py in _batch_retrieve_from_keys(self, keys)
    799                 disable=not self.progress
    800             )
--> 801         for i, kk in enumerate(keys):
    802             if self.print_n_doc:
    803                 print(i)

~/.conda/envs/answer_engine_env/lib/python3.8/site-packages/tqdm/std.py in __iter__(self)
   1164         # (note: keep this check outside the loop for performance)
   1165         if self.disable:
-> 1166             for obj in iterable:
   1167                 yield obj
   1168             return

projects/seal-exp/SEAL/seal/retrieval.py in batch_generate_keys(searcher, queries, constrained_generation)
    308         batches = ichunked(queries, searcher.batch_size)
    309         for batch in batches:
--> 310             for instance in process_batch(batch):
    311                 bar.update()
    312                 yield instance

projects/seal-exp/SEAL/seal/retrieval.py in process_batch(inputs)
     68             batch = {k: v.to(searcher.device) for k, v in batch.items()}
     69 
---> 70             found_keys = fm_index_generate(
     71                 searcher.bart_model, searcher.fm_index,
     72                 **batch,

~/.conda/envs/answer_engine_env/lib/python3.8/site-packages/torch/autograd/grad_mode.py in decorate_context(*args, **kwargs)
     25         def decorate_context(*args, **kwargs):
     26             with self.clone():
---> 27                 return func(*args, **kwargs)
     28         return cast(F, decorate_context)
     29 

projects/seal-exp/SEAL/seal/beam_search.py in fm_index_generate(model, index, input_ids, attention_mask, min_length, max_length, length_penalty, num_beams, diverse_bs_groups, diverse_bs_penalty, eos_token_id, force_decoding_from, always_allow_eos, keep_history, disable_fm_index, sample, stop_at_count, topk, transformers_output, **kwargs)
    482     model_kwargs['use_cache'] = True
    483 
--> 484     decoder_input_ids = model._prepare_decoder_input_ids_for_generation(
    485         input_ids,
    486         decoder_start_token_id=model.config.decoder_start_token_id,

~/.conda/envs/answer_engine_env/lib/python3.8/site-packages/transformers/generation_utils.py in _prepare_decoder_input_ids_for_generation(self, batch_size, decoder_start_token_id, bos_token_id)
    433         decoder_start_token_id = self._get_decoder_start_token_id(decoder_start_token_id, bos_token_id)
    434 
--> 435         decoder_input_ids = torch.ones((batch_size, 1), dtype=torch.long, device=self.device) * decoder_start_token_id
    436         return decoder_input_ids
    437 

TypeError: ones(): argument 'size' must be tuple of ints, but found element of type Tensor at pos 1

[mbevila]

I am going to look into this in a few hours

[mbevila]

@pranavpawar3 would you mind trying out this fork here, before I make a pull request? Everything seems to work fine now.

Edit: https://colab.research.google.com/drive/1TsnQkhwWISfO-HvAc1rNx-NGbqLIdxoz?hl=it#scrollTo=z8Kpyrvt_iP8 might be useful.

[pranavpawar3]

Yes sure, testing it now.

[pranavpawar3]

@mbevila Just tested the fork. All works well! Thanks for helping with this.

[mbevila]

Fixed it in #9.