尽管 StackOverflow 上有很多有用的信息，但我还是不知所措...

我正在尝试将 OpenGL 渲染缓冲区写入 iPad 2(使用 iOS 4.3)上的视频。这正是我正在尝试的:

A) 设置一个 AVAssetWriterInputPixelBufferAdaptor

1. 创建一个指向视频文件的 AVAssetWriter

2. 使用适当的设置设置 AVAssetWriterInput

3. 设置 AVAssetWriterInputPixelBufferAdaptor 以将数据添加到视频文件

B) 使用 AVAssetWriterInputPixelBufferAdaptor 将数据写入视频文件

1. 将 OpenGL 代码渲染到屏幕上

2. 通过 glReadPixels 获取 OpenGL 缓冲区

3. 从 OpenGL 数据创建一个 CVPixelBufferRef

4. 使用 appendPixelBuffer 方法将该 PixelBuffer 附加到 AVAssetWriterInputPixelBufferAdaptor

但是，我在执行此操作时遇到了问题。我现在的策略是在按下按钮时设置 AVAssetWriterInputPixelBufferAdaptor。一旦 AVAssetWriterInputPixelBufferAdaptor 有效，我将设置一个标志以向 EAGLView 发出信号以创建一个像素缓冲区，并通过 appendPixelBuffer 将其附加到视频文件以获得给定的帧数。

现在我的代码崩溃了，因为它试图附加第二个像素缓冲区，给我以下错误:

-[__NSCFDictionary appendPixelBuffer:withPresentationTime:]: unrecognized selector sent to instance 0x131db0

这是我的 AVAsset 设置代码(很多是基于 Rudy Aramayo 的代码，确实适用于普通图像，但不适用于纹理):

- (void) testVideoWriter {

  //initialize global info
  MOVIE_NAME = @"Documents/Movie.mov";
  CGSize size = CGSizeMake(480, 320);
  frameLength = CMTimeMake(1, 5); 
  currentTime = kCMTimeZero;
  currentFrame = 0;

  NSString *MOVIE_PATH = [NSHomeDirectory() stringByAppendingPathComponent:MOVIE_NAME];
  NSError *error = nil;

  unlink([betaCompressionDirectory UTF8String]);

  videoWriter = [[AVAssetWriter alloc] initWithURL:[NSURL fileURLWithPath:betaCompressionDirectory] fileType:AVFileTypeQuickTimeMovie error:&error];

  NSDictionary *videoSettings = [NSDictionary dictionaryWithObjectsAndKeys:AVVideoCodecH264, AVVideoCodecKey,
                                 [NSNumber numberWithInt:size.width], AVVideoWidthKey,
                                 [NSNumber numberWithInt:size.height], AVVideoHeightKey, nil];
  writerInput = [AVAssetWriterInput assetWriterInputWithMediaType:AVMediaTypeVideo outputSettings:videoSettings];

  //writerInput.expectsMediaDataInRealTime = NO;

  NSDictionary *sourcePixelBufferAttributesDictionary = [NSDictionary dictionaryWithObjectsAndKeys: [NSNumber numberWithInt:kCVPixelFormatType_32BGRA], kCVPixelBufferPixelFormatTypeKey, nil];

  adaptor = [AVAssetWriterInputPixelBufferAdaptor assetWriterInputPixelBufferAdaptorWithAssetWriterInput:writerInput                                                                          sourcePixelBufferAttributes:sourcePixelBufferAttributesDictionary];
  [adaptor retain];

  [videoWriter addInput:writerInput];

  [videoWriter startWriting];
  [videoWriter startSessionAtSourceTime:kCMTimeZero];

  VIDEO_WRITER_IS_READY = true;
}

好的，现在我的 videoWriter 和适配器已经设置好了，我告诉我的 OpenGL 渲染器为每一帧创建一个像素缓冲区:

- (void) captureScreenVideo {

  if (!writerInput.readyForMoreMediaData) {
    return;
  }

  CGSize esize = CGSizeMake(eagl.backingWidth, eagl.backingHeight);
  NSInteger myDataLength = esize.width * esize.height * 4;
  GLuint *buffer = (GLuint *) malloc(myDataLength);
  glReadPixels(0, 0, esize.width, esize.height, GL_RGBA, GL_UNSIGNED_BYTE, buffer);
  CVPixelBufferRef pixel_buffer = NULL;
  CVPixelBufferCreateWithBytes (NULL, esize.width, esize.height, kCVPixelFormatType_32BGRA, buffer, 4 * esize.width, NULL, 0, NULL, &pixel_buffer);

  /* DON'T FREE THIS BEFORE USING pixel_buffer! */ 
  //free(buffer);

  if(![adaptor appendPixelBuffer:pixel_buffer withPresentationTime:currentTime]) {
      NSLog(@"FAIL");
    } else {
      NSLog(@"Success:%d", currentFrame);
      currentTime = CMTimeAdd(currentTime, frameLength);
    }

   free(buffer);
   CVPixelBufferRelease(pixel_buffer);
  }


  currentFrame++;

  if (currentFrame > MAX_FRAMES) {
    VIDEO_WRITER_IS_READY = false;
    [writerInput markAsFinished];
    [videoWriter finishWriting];
    [videoWriter release];

    [self moveVideoToSavedPhotos]; 
  }
}

最后，我将视频移至相机胶卷:

- (void) moveVideoToSavedPhotos {
  ALAssetsLibrary *library = [[ALAssetsLibrary alloc] init];
  NSString *localVid = [NSHomeDirectory() stringByAppendingPathComponent:MOVIE_NAME];    
  NSURL* fileURL = [NSURL fileURLWithPath:localVid];

  [library writeVideoAtPathToSavedPhotosAlbum:fileURL
                              completionBlock:^(NSURL *assetURL, NSError *error) {
                                if (error) {   
                                  NSLog(@"%@: Error saving context: %@", [self class], [error localizedDescription]);
                                }
                              }];
  [library release];
}

但是，正如我所说，我在调用 appendPixelBuffer 时崩溃了。

很抱歉发送了这么多代码，但我真的不知道我做错了什么。更新一个将图像写入视频的项目似乎是微不足道的，但我无法获取我通过 glReadPixels 创建的像素缓冲区并附加它。这让我疯狂!如果有人有任何建议或 OpenGL 的工作代码示例 --> 视频会很棒...谢谢!

最佳答案

我刚刚在我的开源中得到了类似的东西 GPUImage框架，基于上面的代码，所以我想我会为此提供我的工作解决方案。就我而言，我能够按照 Srikumar 的建议使用像素缓冲池，而不是为每个帧手动创建像素缓冲区。

我先配置要录制的电影:

NSError *error = nil;

assetWriter = [[AVAssetWriter alloc] initWithURL:movieURL fileType:AVFileTypeAppleM4V error:&error];
if (error != nil)
{
    NSLog(@"Error: %@", error);
}


NSMutableDictionary * outputSettings = [[NSMutableDictionary alloc] init];
[outputSettings setObject: AVVideoCodecH264 forKey: AVVideoCodecKey];
[outputSettings setObject: [NSNumber numberWithInt: videoSize.width] forKey: AVVideoWidthKey];
[outputSettings setObject: [NSNumber numberWithInt: videoSize.height] forKey: AVVideoHeightKey];


assetWriterVideoInput = [AVAssetWriterInput assetWriterInputWithMediaType:AVMediaTypeVideo outputSettings:outputSettings];
assetWriterVideoInput.expectsMediaDataInRealTime = YES;

// You need to use BGRA for the video in order to get realtime encoding. I use a color-swizzling shader to line up glReadPixels' normal RGBA output with the movie input's BGRA.
NSDictionary *sourcePixelBufferAttributesDictionary = [NSDictionary dictionaryWithObjectsAndKeys: [NSNumber numberWithInt:kCVPixelFormatType_32BGRA], kCVPixelBufferPixelFormatTypeKey,
                                                       [NSNumber numberWithInt:videoSize.width], kCVPixelBufferWidthKey,
                                                       [NSNumber numberWithInt:videoSize.height], kCVPixelBufferHeightKey,
                                                       nil];

assetWriterPixelBufferInput = [AVAssetWriterInputPixelBufferAdaptor assetWriterInputPixelBufferAdaptorWithAssetWriterInput:assetWriterVideoInput sourcePixelBufferAttributes:sourcePixelBufferAttributesDictionary];

[assetWriter addInput:assetWriterVideoInput];

然后使用此代码通过 glReadPixels() 获取每个渲染帧:

CVPixelBufferRef pixel_buffer = NULL;

CVReturn status = CVPixelBufferPoolCreatePixelBuffer (NULL, [assetWriterPixelBufferInput pixelBufferPool], &pixel_buffer);
if ((pixel_buffer == NULL) || (status != kCVReturnSuccess))
{
    return;
}
else
{
    CVPixelBufferLockBaseAddress(pixel_buffer, 0);
    GLubyte *pixelBufferData = (GLubyte *)CVPixelBufferGetBaseAddress(pixel_buffer);
    glReadPixels(0, 0, videoSize.width, videoSize.height, GL_RGBA, GL_UNSIGNED_BYTE, pixelBufferData);
}

// May need to add a check here, because if two consecutive times with the same value are added to the movie, it aborts recording
CMTime currentTime = CMTimeMakeWithSeconds([[NSDate date] timeIntervalSinceDate:startTime],120);

if(![assetWriterPixelBufferInput appendPixelBuffer:pixel_buffer withPresentationTime:currentTime]) 
{
    NSLog(@"Problem appending pixel buffer at time: %lld", currentTime.value);
} 
else 
{
//        NSLog(@"Recorded pixel buffer at time: %lld", currentTime.value);
}
CVPixelBufferUnlockBaseAddress(pixel_buffer, 0);

CVPixelBufferRelease(pixel_buffer);

我注意到的一件事是，如果我尝试附加两个具有相同整数时间值的像素缓冲区(在提供的基础上)，整个记录将失败并且输入永远不会占用另一个像素缓冲区。同样，如果我在从池中检索失败后尝试附加像素缓冲区，它将中止记录。因此，上述代码中的早期救助。

除了上述代码之外，我还使用颜色调配着色器将我的 OpenGL ES 场景中的 RGBA 渲染转换为 BGRA，以便 AVAssetWriter 进行快速编码。有了这个，我能够在 iPhone 4 上以 30 FPS 的速度录制 640x480 的视频。

同样，所有相关代码都可以在 GPUImage 中找到存储库，在 GPUImageMovieWriter 类下。

关于ios - OpenGL ES 2.0 到 iPad/iPhone 上的视频，我们在Stack Overflow上找到一个类似的问题： https://stackoverflow.com/questions/6499026/