jellyfin/jellyfin-plugin-reports

Exporting only produces 100 elements

gottsman opened this issue · 4 comments

gottsman commented

I tried exporting the data to Excel and CSV. In both exports, only 100 elements are included in the resulting file. I adjusted the page view to "all" and the resulting exports also produced just 100 elements.

colino17 commented

Having this same issue ever since upgrading to Jellfin 10.8.0+.

Previously I was able to adjust my page view to all items. While this was incredibly slow for items like Episodes it would eventually load and I could export an Excel sheet of every episode in my collection. This was very useful for identifying files that needed to be updated/upgraded/transcoded/replaced with newer codecs/higher resolutions/etc.

Since updating adjusting the page view for all items no longer works and you can no longer export a sheet with all items.

I also tried exporting items 500 at a time. This partially worked, except certain page ranges would not load and therefore could not be exported. In my instance I have just under 12000 episodes. When sorting for page views of 500, every page range loaded except for items 2001-2500. This range would not load, but pressing the next page button again resulted in range 2501-3000 loading as normal.

This is the error that pops up in the logs when I attempt to access the 2001-2500 range:

[2022-08-17 13:50:04.450 -03:00] [ERR] [89] Jellyfin.Server.Middleware.ExceptionMiddleware: Error processing request. URL "GET" "/Reports/Items".
System.NullReferenceException: Object reference not set to an instance of an object.
   at Jellyfin.Plugin.Reports.Api.Common.ReportBuilderBase.GetVideoStream(BaseItem item)
   at Jellyfin.Plugin.Reports.Api.Data.ReportBuilder.GetReportRows(IEnumerable`1 items, List`1 options)
   at Jellyfin.Plugin.Reports.Api.Data.ReportBuilder.GetResult(IReadOnlyList`1 items, IReportsQuery request)
   at Jellyfin.Plugin.Reports.Api.ReportsService.GetReportResult(GetItemReport request, User user)
   at Jellyfin.Plugin.Reports.Api.ReportsService.Get(GetItemReport request)
   at Jellyfin.Plugin.Reports.Api.ReportsController.GetItemReport(Nullable`1 hasThemeSong, Nullable`1 hasThemeVideo, Nullable`1 hasSubtitles, Nullable`1 hasSpecialFeature, Nullable`1 hasTrailer, String adjacentTo, Nullable`1 minIndexNumber, Nullable`1 parentIndexNumber, Nullable`1 hasParentalRating, Nullable`1 isHd, String locationTypes, String excludeLocationTypes, Nullable`1 isMissing, Nullable`1 isUnaried, Nullable`1 minCommunityRating, Nullable`1 minCriticRating, Nullable`1 airedDuringSeason, String minPremiereDate, String minDateLastSaved, String minDateLastSavedForUser, String maxPremiereDate, Nullable`1 hasOverview, Nullable`1 hasImdbId, Nullable`1 hasTmdbId, Nullable`1 hasTvdbId, Nullable`1 isInBoxSet, String excludeItemIds, Nullable`1 enableTotalRecordCount, Nullable`1 startIndex, Nullable`1 limit, Nullable`1 recursive, String sortOrder, String parentId, String fields, String excludeItemTypes, String includeItemTypes, String filters, Nullable`1 isFavorite, Nullable`1 isNotFavorite, String mediaTypes, String imageTypes, String sortBy, Nullable`1 isPlayed, String genres, String genreIds, String officialRatings, String tags, String years, Nullable`1 enableUserData, Nullable`1 imageTypeLimit, String enableImageTypes, String person, String personIds, String personTypes, String studios, String studioIds, String artists, String excludeArtistIds, String artistIds, String albums, String albumIds, String ids, String videoTypes, String userId, String minOfficialRating, Nullable`1 isLocked, Nullable`1 isPlaceHolder, Nullable`1 hasOfficialRating, Nullable`1 collapseBoxSetItems, Nullable`1 is3D, String seriesStatus, String nameStartsWithOrGreater, String nameStartsWith, String nameLessThan, String reportView, String displayType, Nullable`1 hasQueryLimit, String groupBy, String reportColumns, Boolean enableImages)
   at lambda_method2045(Closure , Object , Object[] )
   at Microsoft.AspNetCore.Mvc.Infrastructure.ActionMethodExecutor.SyncObjectResultExecutor.Execute(IActionResultTypeMapper mapper, ObjectMethodExecutor executor, Object controller, Object[] arguments)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeActionMethodAsync()
   at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeNextActionFilterAsync()
--- End of stack trace from previous location ---
   at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Rethrow(ActionExecutedContextSealed context)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeInnerFilterAsync()
--- End of stack trace from previous location ---
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeNextResourceFilter>g__Awaited|25_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Rethrow(ResourceExecutedContextSealed context)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.InvokeFilterPipelineAsync()
--- End of stack trace from previous location ---
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Awaited|17_0(ResourceInvoker invoker, Task task, IDisposable scope)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Awaited|17_0(ResourceInvoker invoker, Task task, IDisposable scope)
   at Microsoft.AspNetCore.Routing.EndpointMiddleware.<Invoke>g__AwaitRequestTask|6_0(Endpoint endpoint, Task requestTask, ILogger logger)
   at Jellyfin.Server.Middleware.ServerStartupMessageMiddleware.Invoke(HttpContext httpContext, IServerApplicationHost serverApplicationHost, ILocalizationManager localizationManager)
   at Jellyfin.Server.Middleware.WebSocketHandlerMiddleware.Invoke(HttpContext httpContext, IWebSocketManager webSocketManager)
   at Jellyfin.Server.Middleware.IpBasedAccessValidationMiddleware.Invoke(HttpContext httpContext, INetworkManager networkManager)
   at Jellyfin.Server.Middleware.LanFilteringMiddleware.Invoke(HttpContext httpContext, INetworkManager networkManager, IServerConfigurationManager serverConfigurationManager)
   at Microsoft.AspNetCore.Authorization.Policy.AuthorizationMiddlewareResultHandler.HandleAsync(RequestDelegate next, HttpContext context, AuthorizationPolicy policy, PolicyAuthorizationResult authorizeResult)
   at Microsoft.AspNetCore.Authorization.AuthorizationMiddleware.Invoke(HttpContext context)
   at Jellyfin.Server.Middleware.QueryStringDecodingMiddleware.Invoke(HttpContext httpContext)
   at Swashbuckle.AspNetCore.ReDoc.ReDocMiddleware.Invoke(HttpContext httpContext)
   at Swashbuckle.AspNetCore.SwaggerUI.SwaggerUIMiddleware.Invoke(HttpContext httpContext)
   at Swashbuckle.AspNetCore.Swagger.SwaggerMiddleware.Invoke(HttpContext httpContext, ISwaggerProvider swaggerProvider)
   at Microsoft.AspNetCore.Authentication.AuthenticationMiddleware.Invoke(HttpContext context)
   at Jellyfin.Server.Middleware.RobotsRedirectionMiddleware.Invoke(HttpContext httpContext)
   at Jellyfin.Server.Middleware.LegacyEmbyRouteRewriteMiddleware.Invoke(HttpContext httpContext)
   at Microsoft.AspNetCore.ResponseCompression.ResponseCompressionMiddleware.InvokeCore(HttpContext context)
   at Jellyfin.Server.Middleware.ResponseTimeMiddleware.Invoke(HttpContext context, IServerConfigurationManager serverConfigurationManager)
   at Jellyfin.Server.Middleware.ExceptionMiddleware.Invoke(HttpContext context)

The same error appears when trying to use the All Items page view.

UPDATE: I've narrowed the issue range to 2141-2160 by using the 20 item page view. Perhaps someone (perhaps the original commenter @gottsman) could replicate on their own system to rule out media differences.

gottsman commented

Seems to be very intermittent. I just tried to export movies and shows. Both were successful for 4,000 movies and 12,000 shows.

After viewing a basic report, I clicked the 3 dots to bring up the window on the right. Clicked all and Excel. It spent some time chugging and eventually brought up the save dialog. The file opened fine

DareFox commented

Exporting all episodes (200) in CSV was successful, but for Activity I got only first 100 out of 3645

sunjaxx commented

Same here... Exporting Activity only exports 100 records...